Every time I stop reviewing the code from Claude I later find janky crap. Not every review finds janky crap, but often enough that I can't stop reviewing everything.
This is my disconnect with this kind of speed, how are they avoiding the janky crap? Or are they just accepting it and hoping it won't be too crappy or be found soon enough?
I don't have any insight into Anthropic but it seems like that's the case which is just depressing.
They're just accepting it. It shows in the quality of their products also. They ship a lot of features in Claude Code for example but it is clear that is a vibe-coded mess.
Mess might not be the right word, but lots of features are clearly developed without any grand vision or coherent thought about how the features are supposed to work together.
Haha, that's a good one. Perhaps we should coin a new phrase: "dog shitting", where a company eats its own dog shit before inflicting it on their users. :)
Did you hear the stats on the claude code leak? there were multi-thousand line for loops with hundreds of conditional / if statements written in typescript. Its insanely difficult to decipher. At this point "mess that only AI can understand" is what they have and must continue building upon.
Also, we don’t know how many billions of dollars was spent on making it work. They are running 1000s of agents against their codebase 24/7, and still only producing what they’ve put out so far.
It might be fully automated and eventually the agents figure out all of the bugs, but I suspect the costs are so large that no sane company would ever write code this way, and probably WAAAY more expensive than just hiring humans to write and test it.
+1, even the strongest models still produce things that (I suspect) if it accumulates over time will cause issues.
Is my suspicion wrong? Are models getting better fast enough that they'll be able to add new feature (or fix bugs) and clean up even despite a mountain of AI tech debt [1]
[1]: I say AI tech debt because it tends to have some characteristics that are different from human tech debt, such as apparent lack of intentionality (which makes Chesterton's fence evaluation much harder). Perhaps if we make the AI write down somewhere, as briefly as possible, why it wrote each chunk of code...
> Perhaps if we make the AI write down somewhere, as briefly as possible, why it wrote each chunk of code...
Well-configured agents do, in comments and commit messages and docs.
But many people are just stuck thinking in GPT-3 era, where the prompt was all there was to it. It's not. You write a 1000 token prompt describing what you want, and the agent pulls in 10k tokens of system prompt, then spends time exploring the codebase and calling tools and attending to 100k of more tokens from this, possibly including output of sub-sessions that do the above, and somewhere along the way, having read lots of code, run lots of tests, perhaps doing a spike or two, the agent then decides some chunk of code needs to be written in this way, and not in some other way.
The reason is not contained in your original prompt at all. It's almost entirely based on what the agent learned about the current state of your codebase and environment.
This. You’re no longer prompt engineering you’re now building the factory. There’s lots of tools to do this - rules, skills, hooks, subagents, evals, hill climbing etc. highly encourage everyone to think about building the factory that builds the product, not building the product with ai assistance. And it’s fun again.
tl;dr - With AI code is cheap, but poor architecture and skipping best practices is very very expensive. Markdown files will not solve this problem, but it can be solved.
In my case, “Factory” is hundreds of architectural tests built over thousands of application specific tests, and a couple MD files here or there. The MDs are mostly reference material and common patterns. The tests are what actually enforce feature/module isolation and proper architecture.
Lately, I’ve been getting overzealous with protobuffers and service discovery as well. I would never use these tools so frequently if I was designing for humans. With AI I’ll slap annoying boundaries in to enforce separation of concerns without thinking twice.
Making a new project is a pain at this stage, but you can just point to CI and the general structure in an existing project and AI will clone the important bits. I suspect high level architectural rules with programmatic enforcement will eventually be encoded into tools for easily starting new projects. Kind of like pnpm installing specific development practices and architectures.
I used to do a lot of work with reflection, compile hooks, and dynamic source generation before AI. You can do a lot to compensate for the architectural blind spots of AI if you get creative.
> You can do a lot to compensate for the architectural blind spots of AI if you get creative.
How about code blind spots like endless code duplication? Reading the whole DB to read one record? Slow algorithms? Bad data access patterns? Excessive defensive programming? Inventing new issues just because your "high level architectural creativity" mentioned something off-hand? Etc. etc.
On top of that, where are your "hundreds of architectural tests built over thousands of application specific tests" coming from? Also written by LLMs I presume?
> How about code blind spots like endless code duplication? Reading the whole DB to read one record? Slow algorithms? Bad data access patterns? Excessive defensive programming? Inventing new issues just because your "high level architectural creativity" mentioned something off-hand? Etc. etc.
All of these are endemic problems in this industry, openly discussed for decades before LLMs were a thing.
But the pragmatic reality is the same for people and agents - those "mistakes" are common; juniors make them because they don't know better, seniors make them for good reason (pragmatic tradeoffs). Good teams know how to prioritize, and generally fix them when they start causing problems. Agents are very much capable of the same, fully autonomously these days.
Personally, I've had plenty of cases where an LLM came out with a sleak, and arguably over-optimized algorithm or DB query - but when I looked closer at what it did, I saw it first came out with something dumb and claimed a win, then run it, realized it's stupidly inefficient, and optimized it on its own, before giving the final version to me. All in a single turn, without being explicitly prompted to work like this.
And then it reads all the AI-written "docs" from the previous changes, at which point it's all so muddy, it just tries to copy the style and output more junk and more verbose "docs". This is why I never let AIs maintain documents on their own.
And agents that fail to comprehend the context correctly get stopped quickly.
--
But let me spell out my original comment, to clarify:
Agents are like interns: they come to do the job, having none of the knowledge specific to your codebase and company. They have no time, to properly study the whole thing. They get a task, and are expected to figure it out (with help of documentation and people). Then they do the thing, and regardless of the outcome, are gone few months later, and you never hear from them again.
(In many companies, for various reasons, juniors are effectively interns with larger context window.)
In large companies with large projects, everything is moving so fast that no one - no one - can possibly understand the whole thing, and the work paradigm is purposefully structured so they don't need to. In some sense, they're like perpetual interns - each time they start working on different area of the project, they know next to nothing about it, and do not have a chance to fully comprehend it. They just need to figure out enough to deliver on their tasks.
(I know, because I like understanding things, and in one place I was told explicitly it's a common trap for people like me, and I need to learn to suppress it if I am to be effective at my job.)
Unsurprisingly, this works. Both examples are proof positive that this way of working is sustainable and scales well. It may not be optimal, but it works, which means agentic version of it has every right to work and scale as well.
Literally nothing in this wall of text has any televance to what I wrote. And it has a bunch of baffling statements:
> Agents are like interns: they come to do the job,
They don't come to do anything. Stop this infantile antromporphising of LLMs.
They are nothing like interns. Interns learn. And companies don't develop products by getting a new batch of interns every day.
> Unsurprisingly, this works. Both examples are proof positive that this way of working is sustainable and scales well.
No, it's not even close to proof. The conclusion literally doesn't follow from anything in your prevuous text.
If your agents are "perpetual interns" then this is proof that no, it doesn't work and it doesn't scale.
As for large companies... You don't have to know the whole thing, but you do have to know your area. You don't come into work at a large company always asking "what is it I am doing" from scratch every single day.
Replace "day" with "three months" and it should click.
> They are nothing like interns. Interns learn.
And then they go elsewhere, so no, from POV of the company, if you treat "internship period" as "session", they don't learn. But they leave behind documentation and changes to code.
I guess another way to put my point is: the idea of code being self-documenting isn't completely flawed. It works to a degree; in fast-moving teams in large companies, you don't have any other ground truth anyway (documentation is always stale, design docs become obsolete the day they're approved, etc.). People manage. And so can agents.
> but you do have to know your area
It so happens that LLMs know nearly every single area of human endeavor better than most practitioners :).
> Replace "day" with "three months" and it should click.
No, it shouldn't. If you compare agents to interns then you have a new batch of interns not even every day, but every few hours (with every new session, and every new subagent).
> so no, from POV of the company, if you treat "internship period" as "session", they don't learn.
So,
- interns are a very tiny fraction of a workforce at any given company
- no company is ever building the bulk of their products with interns alone
- from POV of the company interns never learn, and the new interns should be taught everything from scratch, again
Somehow, from all this, your conclusion is that "it's proof that it works and can scale"
> It so happens that LLMs know nearly every single area of human endeavor
They have very approximate knowledge of those areas + this knowledge is rather randomized in the output. On top of that they are trained on publicly available code which is shit. Plus they know nothing (and can never learn anything) form your particular code, and your particular project.
Nope you are 100% correct. All of the models will code themselves into local maxima then spend more and more time chasing their tails as every change introduces new bugs, the agent adds more complexity, and the code base just spirals. Best case you end up with a bunch of parallel implementations that all work slightly differently.
It helps a lot of you pay attention to what it is doing and impose order, then regularly spend time refactoring - even if you are having claude do the work itself - you have to spend the time on the overall architecture and design.
I just experienced this with an internal productivity tool. Without claude this tool wouldn't exist - there would never be the justification to take the required time to build it. On the other hand all the interlocking refreshing rules for sourcing data from its inputs got so byzantine it was impossible to add new features or change anything without causing new regressions. Claude built a web of ad-hoc timers, refresh kick-offs, monitors, etc to solve each feature problem or fix some bug but the whole thing was impossible even for claude to understand. It took a special large workflow session to find most of the mess and even then during the refactor it kept finding new things.
I had to stop all feature work for going on four weeks and burn a ton of tokens to build a refreshing/syncing engine that could track dependencies, priorities, etc and fully test it and just now today I am finally able to get feature work going again.
On another feature an agent came up with a state object containing 55 inter-related fields with all kinds of interlocking dependencies. Something so complex no one could hope to understand it. And I find the agents aren't really much better at understanding such things than humans are. If you leave it like that the agent will just perform worse and worse on that part of the code as time goes on. It will make assumptions that force it into bad design corners, miss edge cases, and flat out miss things no matter how well you prompt. Agents benefit from encapsulation and simplicity just as much as humans.
As others have already mentioned, they don’t. Even something as important as payments they seem to care little about, couldn’t give them money for a while due to known issues with their web.
But also, anything coming from Anthropic (and the rest) is 100% marketing.
Many of us have used Claude Code. And Codex too. These tools are essentially produced by bots. What is the quality like? I would say pretty good! Are there occasional issues in the tool itself? Yes. But more than average software produced by humans? No. Not at all. In fact I would say these tools are significantly better than average commercial software.
Pretty good? The list of bug fixes that's shipped for Claude Code every day doesn't fit on my screen. Their issue tracking is a big mess with 100.000 issues in it, most of them auto-closed. People even (vibe)coded websites to track if a new build is usable (e.g., https://claudeissues.com/status) that my Claude now started using to explain its own behaviour.
Your bar for quality must be pretty low if you feel this is better than average?
Maybe they're actually using the model to review and clean up the code too? I hope you realize that "just merge whatever the LLM spits out on its first attempt" is not the only way to use LLMs.
Have you used any other harnesses? I feel like the shortcomings of one harness may not be visible until you contrast it with another that does the thing correctly.
My favourite is Claude Code's /rewind feature, which gets so many things wrong for having such a simple UI. (Does selecting the default option still wipe your filesystem changes by default when going back to a previous entry? I found a poorly-documented way to circumvent that "feature", which only adds to my initial point.)
That's easy fixed by actually taking the effort to go and use other AI harnesses. Youll see that Claude Code somehow works then it doesnt for no obvious reason, then again something else... Like there is no system to it. Its somehow obvious that it isnt developed by humans or at least not as enough as should be.
They spent a year trying to make a TUI renderer that doesn't flicker. They finally shipped a replacement, and there are still baffling bugs in it three months later. Like the difference between what it renders when you paste something into the input box, and what it shows after submit
Theyre usable but leave alot to be desired. SOme other tools like opencode seem sometimes to have less feature (if exact numbers) but at least always work.
Somehow with Claude and some others, random simple thigns dont work. Esp long solved problems like TUI rendering, special chars, breaking changes every release etc.
Surely as always with software this is my experience and might be not yours.
PS: Ive used IntelliJ since... always?. It was quite decent. But these last years it is just so diferent, somehow way less quality, so many random bugs that Im afraid theyre doing AI agentic coding which sucks for such an important tool.
IDEs should be build from real professional crafstman with an exact vision in their heads, not some wild AI.
Im starting to think in a year or two "We dont use AI in developing our products" will be some kind of marketing trick since people will start looking for that.
I think at work I am just barely in the category of I've removed more code than I have added. Over many, many bugs and long meetings I've sort of arrived at the conclusion that in most cases, the best code (excluding comments) is the code that doesn't have to exist [0]. It occurred to me at work today while refactoring some change that even in this LLM-driven world, I can still optimize for that for some additional challenge and satisfaction.
The root of this challenge comes from more text aka more code ~= more tokens, so these firms have little incentive to optimize for that mindset. Sure, they might have more optimized individual responses - but the end result is still giving people what they want: more features, more documentation, nice, long, verbose and well-formatted Jira comments, of sometimes questionable but sometimes undeniably impressive quality. The band marches on, and when your codebase outgrows the constraints of the tried-and-true tools, you can just invent your own. With documentation and an MCP or two, or ten -- to also pull into your context window.
[0] I work on an operating system, so a suggested retort is to say this is me pushing responsibility and logic to the people higher up on the stack ;-)
> The root of this challenge comes from more text aka more code ~= more tokens, so these firms have little incentive to optimize for that mindset.
I'm not sure if this is true. The code is such a small part of the total output. Thinking harder, like what would be needed for architecturing better code, would use more token.
I mean, I've mostly switched from Claude to Codex because I find the harness so buggy and annoying that I avoid it when I can. So.. as an (almost former) customer I don't think they're avoiding the issue at all.
I am experiencing the same. I have decided that the only way to use these tools (at least for now) and at the same time to avoid the janky crap and understand what's going on with the codebase, is to ask from the model to work in very small committable batches which I can review without my head exploding, commit in branch, repeat. I feel much better now and I catch things fast and early.
This has become my approach as well. Create a work tree per larger thing / batch of things and manually review everything in small commits, then squash merge the whole tree to keep the history manageable and clean.
> This is my disconnect with this kind of speed, how are they avoiding the janky crap? Or are they just accepting it and hoping it won't be too crappy or be found soon enough?
Using Claude code for a few days gives the answer pretty quickly: they fully embrace the janky crap and ship it to their users without second thoughts. And as long as their users don't mind using their piece of shit, why should they do otherwise?
Engineering is about finding the best solution for a problem given time and money budget constraints.
Not throwing endless resources and energies at a problem "because Twitter told me what the best practices are".
E.g. Maybe Claude Code could use half the memory, maybe, but maybe the correct estimation is that solving the memory problem would negatively effect development speed, or would cost many millions of engineering effort and no real world user would really care whether it's using 50 or 500 MBs of ram.
So no, you're not an engineer but a "developer", if you think that "quality" code is engineering, code has a development and maintenance cost, specific directions come with important trade offs.
But since it isn't (we know real users care and complain, we know it's not your made up "endless resources" and "millions of engineering effort" because "millions" of other software products exist that have been made without those) you're short of an actual argument.
> code has a development and maintenance cost, specific directions come with important trade offs.
Yet again you forgot to make a case how any of those trade off towards the current state.
In general, how would you ever determine whether the current situation is justified or not if nothing in your strawman questions anything, but just comes up with not very plausible justifications of whatever already exists???
If by this you mean that engineering is about meeting specifications. If by this you mean half assing some janky shit that barely clanks over, then no.
Engineering is the science of quantifying risks and costs. If you're not able to quantify the important aspects of a design before constructing it, it's not engineering.
I don't think people having the inner desire for excellence would work in the kind of environment that is Anthropic, let alone ship AI slop (as opposed to nice code, AI generated or not).
The process is actually being broken down into enough small steps that there's less misalignment and drift.
There are also some pretty detailed videos out there on YouTube of Anthropic employees showing how they are using cloud code on several orders of magnitude better than where the bulk of usage is.
How can you measure this? It's a cli where I type text in, and the tool fires it off to the model and runs commands locally. The model might be slow, but how would I notice if the cli was slow? Serious question.
For a while just having an indle conversation open in the background would make my PC go very hot. This seems fixed now but that's definitely not the only performance issue I've faced.
Also, the fact that they managed to leak their own code base, twice, tells you all you need to know about software quality at Anthropic.
The shiny lighting animation that runs on the spinner verb runs at an abysmal fps. 20 fps max, and it stutters quite often for me. Not important functionality wise, but it is quite ugly.
it seems rude so i dont bring this up when a coworkers tell me their vibe coded prototype is working and theyll never look at the code, but how does someone ignore that they charge per token?
as long as something works "good enough" and the code will never be reviewed theres only incentive to blast through tokens generating superfluous code. god forbid they actually ship a vibe coded product then they can basically ransom you for tokens
im probably being overly cynical but it sort of just seems like standard capitalism to me
Most of the world isn't perfect, and trying to be perfect means you get nowhere. In most places, excessive perfection is implicitly avoided through resource constraints.
In the end, what matters is whether something fulfills its purpose, not how it does it. Code quality, understanding code, etc. are just a means to this goal.
Some level of poor quality is acceptable, and I assume they are hoping that model development (i.e. its capability to fix and clean up an increasingly convoluted mess) will outpace the tangledness of the code.
So far, it seems like they are winning. Yes, the UI is often broken (e.g. for a long time switching away from an ongoing conversation lost it), but it wasn't broken enough for people to switch to other companies that may have had a better UI but in exchange for that a less developed product. If Anthropic instead spent its resources on making the UI not janky and code reviewing everything, the overall product would have likely been worse because they couldn't build and improve features as quickly.
I've seen the flip side where "code quality" becomes its own independent goal and more time is spent gratifying various linters than writing the code, and the inability to get anything done in such an environment is a lot more depressing than janky UX that can probably be fixed by prompting the next generation model to clean it up.
Code quality - and its best friend, interface design - pay dividends in large codebases. The fastest moving and most effective organizations I've worked at are ones that focused on code quality from the beginning.
All software is broken, but LLMs seem to make the wrong compromises consistently when working in large codebases.
Now instead of the AI agents swarm you let 100 engineers making the changes on that large codebase. How do you keep the code "quality", which is a very subjective thing, while at the same time, as you say, keep the "fastest moving and most effective organizations" trait? Impossible without gatekeeping and thorough review process both of which contradicts your trait.
Good interface design allows extending or changing component X without having to step on the toes of your colleague working on component Y downstream. It's absolutely essential.
Interface design is one of the things Claude is worst at in my experience (using it every day for the past year).
But a little bit of directing of the architecture can easily bring in design patterns.
Ideally, you design patterns that enable quick replacing of parts that get to messy. The code is cheap, just start from scratch with the defined functionality pre defined instead of bolted on with jank.
That reads as an example from GoF era. In practice, especially with complex software engineering codebases, which I thought you were implying by saying "large", these patterns barely exist because the problems being solved are often in friction with what is considered to be the "best practices" elsewhere (in the books, other much simpler codebases, etc.).
Thank you for engaging with my point! I agree, actually. I think if good design patterns could simply be copied out of a book, LLMs might be better at software engineering.
My point is more about up-front investment paying dividends. To speculate, it seems like the reward hacking is not there for long-term thinking.
I would argue that the "fastest moving and most effective organizations" trait is relative to what we find in the set of software develepment groups. There must be a difference in code quality and development speed between the best and the slowest player, and there must be some parametrizable difference in standards (if there are any) the players operate with.
This is too vague for me to answer but I'll try - effective organization is the one whose revenue and net-profit per employee is higher than their competition. And fastest moving organization would be relative and tightly coupled to being an effective organization too. Reminiscent of a fail-fast culture without red tapes. Software "quality" is IME rarely a discriminator.
It's true though. I've also experienced the difference between a codebase that was an absolute tarpit from the start, and one that obviously focused on quality from the start. They both became jankier over time, but one notably less so. The latter was a bigger, more successful organization. Does that have anything to do with their codebase being cleaner? Maybe. Maybe not. It was certainly easier for me to work in.
> The fastest moving and most effective organizations I've worked at are ones that focused on code quality from the beginning.
I agree with that in the pre-LLM world, but I don’t believe it’s the same today.
You can build something with “bad” code that works and is full of technical debt, but today paying this debt is getting cheaper and cheaper.
So if Claude’s wrong design decision leads to having to refactor several chunks of a codebase in order to implement a new feature, does it really matter if it’s Claude who is going to refactor it in a matter of seconds? Especially for a company like Anthropic which has a virtually unlimited number of tokens to spend on it.
This is a fallacy. Computers are very deterministic, a well designed program will not have any bugs or unforeseen failures at all. You're not planting a tree and going "oh god I hope it survives the winter", you're programming a human-made computer, made of hardware that undergoes such incredible scrutiny before it's ever powered on that you can trust that it executes the same code, the same way, every single time, for decades.
No idea why people who clearly don't understand the difference between poppy farms and computers come in here and claim the world isn't perfect and so we should all accept sloppy shit code that barely works. The world isn't perfect, but software CAN be quite close.
Every single instance of jank on computers is a bug that someone wrote, or an LLM wrote, that can be found, understood, and fixed. And only once you understand this, can you talk about software quality to other engineers. You need to understand that the baseline is that software is not fallible the way the rest of the world is.
See sqlite3 if you want to know what nearly bug-free software looks like. I can't point you to a potato farm or an artist who is perfect, but I can point you to that and say "this is where we go when humans do a good job".
> This is a fallacy. Computers are very deterministic, a well designed program will not have any bugs or unforeseen failures at all.
I don't know where you think you've found a fallacy. Computing systems are expected to interact in various ways with the outside world, which does include poppy farms, and people, and other decidedly imperfect things.
Yes! But they can behave deterministically for any kind of input data. Of course they interact with the outside world, and I'm not saying you can't have errors, I'm saying they're not random and driven by god's will, they're programmed explicitly.
Unless the chip has a bug or manufacturing error; I've worked with several fleets where small numbers of chips had stochastic errors. I doubt there is any fleet which does not periodically have some level of corruption, which typically goes unnoticed unless you have engineers paying close attention and some level of verification.
In my first class of Software Engineering, I was taught that software is designed for changes. Even if software the fully deterministic, it interfaces with a world that is constantly changing: requirements change, customers change, political systems change.
I want to agree with you, but most software projects people work on are ridden with unknown bugs and people do in fact live with this. I'm not aware of any commercial software project that arrives at the goal "not having any bugs or unforeseen failures at all". And where this does happen (space probes?) it is very expensive.
IMO the issue with LLMs is they massively widen the "unknown unknowns" (skipping review makes this problem even worse). This is obviously much worse when an LLM is actually a part of the software rather than just producing software that is otherwise deterministic.
Exactly! That's my point. It's already a problem, and it's not "fine" just because it's the status quo, but at least we can chalk it up to "humans are fallible". LLMs 100x the amount of mistakes a programmer can make in a day, and that's inexcusable to not take seriously, that's my point. We don't need to chase perfection, but we can't pretend like whatever Claude one-shots is any good at all.
LLM is clearly not 100xing the mistakes as we have witnessed Anthropic’s success. Otherwise the system would not have been functional. It’s not like their system is not tested.
The thread is about jankiness, lack of feature cohesion and code quality.
What we are discussing here is: is it acceptable to accept poor code quality and trade it off with delivery?
> What we are discussing here is: is it acceptable to accept poor code quality and trade it off with delivery?
I found that people that lean towards the latter also favor profits over customer satisfaction. Then they will try to defend themselves with a black and white view “it’s either this scrap or no delivery”.
> I'm not aware of any commercial software project that arrives at the goal "not having any bugs or unforeseen failures at all".
It's ludicrous that we have executives at tech companies that think this is possible.
One S&P 500 company I worked for signed a big contract to deliver bugless software for an enterprise web portal for the state of Minnesota. I was on the team that had to make that magic happen. I had to leave the company not long after that due to nearly life-ending burnout.
So I would throw scorn at the ivory tower inhabitants here who should know better than to suggest that anything of substantial complexity can be bug free
> a well designed program will not have any bugs or unforeseen failures at al;
You can only say that of either a very simple program or perhaps one with formal verification (which is overkill for 99.99% of software that gets written).
Look at the Linux kernel, arguably a well designed system (unless you ask Tanenbaum) with top notch developers, and still is full of bugs and security holes.
I agree with your general sentiment on bugs. Id personaly go one step further and say:
Every system in the world has bugs and security holes. If its exists, its buggy because "bug" is very loosely defined as "doesnt exactly work as expected". The security topic is even more complicated.
But I also would like to point out that comparing the Linux Kernel quality and development to Claude Code is surely a joke. The former is an ocean of features, compatibility and history. The latter, in the end still one executable CLI tool calling endpoints.
I have never in my career seen a (useful) program without bugs. I'd go so far as to say that, at least for most software, the optimal amount of bugs is not zero - if you write your code carefully enough to never have bugs, you will ship far too slowly. The point of code is not to be beautiful and perfect and well designed - it is to solve problems in the real world.
By the way, all hardware has bugs too; we just don't see them often because the programmers building the stacks that we run our code on work around them.
This is a category error. "memory" is not a computer, so the existence of ECC proves nothing about computers. Concrete is not deterministic in the real world either, and we're still able to control the shape of structures and make them last for centuries.
> Most of the world isn't perfect, and trying to be perfect means you get nowhere.
That's a thought-terminating cliche. When someone asks how can they can avoid generating a pile of slop (which will compound over time and get increasingly harder to deal with), they're not seeking "perfection", and framing it in those terms is dishonest.
Furthermore, can we step back for a second and remember that user and developer experience matters too?
Working with code like that sucks. Spending much of your day on polishing a turd that grows more unwieldy with every iteration is going to get old, fast. And I also think that's going to take a significant mental health toll on people eventually.
The business won't care about this but we should. AI was supposed to make things better, better quality, better reviews, better QA, better product, better UX - remember?
> janky UX that can probably be fixed by prompting the next generation model to clean it up
I'll counter by pointing out that "Next one will fix everything" has been a meme for at least 2 years now and the slop just keeps getting sloppier.
+1 my code quality degenerates exponentially after every prompt if I skip reviewing
Current agents never take a step back to refactor and to create a good abstraction as the requirements evolve, they only care for delivering fast even at the price of garbage code which doesn’t scale
An idea to start could be something along the lines of an agent given tasks incrementally, where the agent would be required to extend/generalize a piece of functionality. A possible reward could then be the amount of code the agent had to add/remove to complete the task.
I assume frontier labs will start hiring devs specifically for generating training data for this purpose. It's a skill you can get better at. I don't see in the limit why it couldn't be solved, but I agree, for practical reasons it's somewhat unlikely.
> I assume frontier labs will start hiring devs specifically for generating training data for this purpose. It's a skill you can get better at. I don't see in the limit why it couldn't be solved, but I agree, for practical reasons it's somewhat unlikely.
Because of the scale that would be needed. Like, design stuff is much fluffier than code correctness or functionality, so really really hard to generate enough synthetic data to tune the models to do better.
In the limit yeah it would work, but in the limit we're all dead ;)
At some point on this ride, if devs are really put out of work and wages are massively depressed (along with the devs), then it might become cheap enough to make work.
They almost certainly don't review the generated code, or at least not with the common techniques we used so far (human review), otherwise their development would be largely bottlenecked by the review process - and I imagine this is where 99% of the companies are at right now.
While companies are heavily incentivized to speeding up, cutting costs, etc. I expect that friction imposed by the humans is not going to go away that quickly.
Does anybody know of "Real" companies that dropped reviews due to immesne use of AI agents, successfully, and their products got visibly better (or at least not hellish worse)?
By "Real" in this context I mean companies that aren't Anthropic .i.e. don't sell any kinda AI product and existed at least 10 years before the AI boom.
Esp. in regulated or at least sensitive fields like finance, banking, medical, logistics etc.
Dropping human-in-the-loop reviews is inevitable - it's just a matter of time. You cannot design a Ferrari and then let the driver crawl its speed by manually pushing it down the road. And all the "real" companies I have worked with are not 100% but 1000% into the AI - the incentive is huge.
Code quality is just going to get worse. I think that ship has sailed. Its no longer economical for humans to write good code. We're all going to be building better scanners, automated review mechanisms, quality checkers, feedback routers, etc. for the next 5 years.
They aren’t. Their source leak showed what a mess of a codebase they’re working with. That was, I assume built with unlimited access to frontier models as well.
Ultimately the "speed" that matters, is the speed of delivering _useful_ improvements over time. The speed of delivering that today, is bottlenecked by the speed users can accept and form opinions on new functionality, and the speed of doing it tomorrow is also bottlenecked by compounded technical debt.
For now at least, I'd say churning code faster and faster cannot widen either of these bottlenecks much.
Today, I use Claude heavily, but review all the code. I'm faster, but not by the multiples people speak of. And that seems ok to me.
Its funny the worst code I used to notice was my old code that solved the problem I had faced at the time but had some flaw in the thought process that I later corrected. The same thing applies to generated code. Yes there is a lot of garbage but you can figure out ways to generate better output as you learn the tools and the models get better
In my experience, the worst crap gets into the codebase through updates. Especially when dealing with something hard, Claude will take shortcuts, that undermines the project's quality. In my projects, I deal with it by still pair programming with it. But as far as I understand, Anthropic deals with it by re-creating projects, rather than updating them.
Maybe the solution is just to hire someone who likes to fix janky crap full time and have that be their job. Honestly, I'd do it. I hate jank, so I like de-jankifying things. I don't think I'd mind such an arrangement personally.
I guess, they have some form of harness engineering optimized for Claude + they likely use more advanced models than those available to us.
And yes, they likely accept that not every line of code is clean.
But honestly (and unfortunately), average source code is nothing stellar either. I do SW dev for 30 years. And most projects I have seen were somewhat ugly.
> Every time I stop reviewing the code from Claude I later find janky crap
I'm very curious about that because that has not been my experience at all. I used to find janky crap from older models and I had to review every line of code very carefully. With newer models, and especially large ones like Fable, it just doesn't happen anymore. The code that I get is what I would have written. Actually, it's usually better because it covers more edge cases that I wouldn't have thought about.
At this point, I guide Claude Code, provide general direction, and make design decisions, but I rarely write code anymore. The code that I get from the LLM is just better.
I do not fully understand where the disconnect comes from. I do believe people that say that they get crap from LLMs, and I suspect I can guess some of the reasons for that, but I cannot be sure.
> I do not fully understand where the disconnect comes from. I do believe people that say that they get crap from LLMs, and I suspect I can guess some of the reasons for that, but I cannot be sure.
Same here. There is an introspective comment below by khazhoux asking the question from the opposite experience. Why do developers end up on different sides of this coin?
LLM capability is obviously important variable. But those with bad experiences report using top tier models, so that isn't it. Likewise, level of experience can be ruled out. Each side of the coin have experienced developers reporting.
A lot of the other variables are not reported but I can go from my own experience.
Greenfield vs brownfield? The first is indeed usually easier but I get great results updating large code bases with decades of development. And I suspect developing greenfield with LLM is easier as the code is better written from the start.
Rarity of the subject matter? In principle, yet another XYZ app or web page is well represented in the training data so the LLM should have an easy go at making similar. In the past with low-end LLMs, this was noticeable. Working in relatively obscure programming languages produced just garbage. But, that was prior to the inflection point. And today I get good results working on relatively arcane scientific/mathematical algorithm as I do when I ask the LLM to barf out some web app.
Different levels of testing and validation and expectation? Even best LLMs make mistakes. The better LLMs make fewer and expose fewer still to the human as they create internal tests. Maybe people have different expectations and those with bad experience expect perfection from a single prompt? Perhaps, but any experienced developer knows bugs are a part of software development.
What else?
Whatever the answer, it's not a simple one and it may be a confluence of many smaller things. Maybe different developers have a multi-variant function in their brain that pops out a different quality-of-software score and also a different threshold that divides good from bad and all we see in their reports is that binary conclusion.
I think it is mostly this and my experience is that agents feel much more at home here. But on the other hand monorepos disadvantages become even more pronounced with agent work. It will become an unmaintainable mess, even if your agent can iterate through it regularly.
I still think building a pipeline that allows for separation of concerns is a requirement for maintainability. Sure, Claude excels to build software if I just give him a server where he can run wild. Any new requirement is an extension of the old repo.
But at some point you just have a giant mess and I am not really convinced of this approach, even if it certainly fits agents.
> how are they avoiding the janky crap? Or are they just accepting
They're obviously just accepting, have you seen any of their results? No internal insights required for these observations. Check out their clauded home page - try to open the right hamburger menu and click on 'policy' - it's impossible to do from the top because "Research" will expand on hover and push 'policy' down, and when you move further down, 'research' will collapse and 'policy' will jump up.
Likewise, the AI apps are infamous for their poor performance and buggy nature. Correctness is the price of such velocity
on mobile it works (where I would expect the hamburger menu), and on usual desktop I have the usual menu bar, but when I change the window size on desktop I get the hamburger menu (and then the UX jankiness is evident, but for me easy to work around, as moving the mouse down closes the "research" block and then it's quite simple to move it up to be able to click "policy")
at this point I think Opus does something better by default
why wouldn't it work on mobile? As you hover your finger over the screen, the finger's shadow doesn't generate any events for the buggy website to react to?
By the way, the hamburger shouldn't even be there as there is enough space to fit the full menu (and even the company name) at the viewport sizes I've checked
as I was reading your comment I was thinking about what the bug is, and my initial guess (pulled out of thin ass-air) was that even on mobile somehow the touch targets overlap and there's no way to actually send the click to "policy"
They get the latest models, they probably don't use the quants they give us, they have unreleased models, etc. We're not in the same situation. Maybe people who use the API idk
> This is my disconnect with this kind of speed, how are they avoiding the janky crap?
My understanding is that they have manual reviewers in phase 1 of their projects. Their goal is to refine the automation of reviewing, writing skills/rules, etc, enforcing architectural choices, etc.
Then in phase 2, the review phase is AI-driven itself and humans only get to be decision makers of escalated issues.
I've been building some game user interfaces with it, and on the surface it looks amazing. It's iterating at lightning speed, but then after a while you start noticing all the holes.
For example I've made a first screen with a lot of elements, and now I want to create a 2nd similar screen, so I tell it to use the same code as we've created for the first to make the 2nd. With that as input it will create it and
a) use e.g. the variables and setup from the first
b) copy everything into the 2nd
c) copy some things
d) make a generic parent for both to inherit from
A human would have realized that this is something we're going to be doing a lot, so the last is the correct answer. Make generics for all elements, e.g. the screen's panel, effects, buttons, text etc. then implement those along with possible overrides etc. for all.
But it seems that it has no clear methodology. Depending on your exact wording along with today's weather you will get any combination of the options.
People need to actually put in some effort with LLMs here. Ask any SOTA agent to identify a specific type of "janky crap", and it'll quickly find every single occurence. Fixing it with the right pattern should be trivial for an experienced engineer, and therefore its agents.
Just do this for every type of "janky crap" and your codebase converges to perfection.
It's not that hard, but it takes a lot of tokens, and setup.
“Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.“
Probably is a source of some bugs. But probably what they’re seeing is that it isn’t a major factor. Rather a bigger factor is agents trying to make changes were the intent isn’t aligned. And note, they don’t say they never look at the code. I suspect that valued human experience is having a feel for what types of changes is AI more likely to make mistakes with.
Many of the comments here seem to stem from the implicit assumption that before claude, software products were thoughtfully designed, secure, reliable, and beautifully engineered works of art.
If you spent some time reading Anthropic's docs on their website, you'll notice that sometimes the same feature has different intended behavior depending on which page you are reading and may or may not match the actual software behavior. You are better off using the docs as inspiration and figure out what the software does at runtime. And I have definitely done the thing where I just ask Claude to reverse engineer its own binary. I doubt anyone at Anthropic can explain what it's supposed to do.
I don't think too much of an insight is needed to understand that most of them 30000 code changes per day is just going around in circles, agent A introducing some kind of new abstraction, agent B finding and refactoring that abstraction, agent C figuring out that refactored abstraction introduced regressions, fixing and creating new tests for an abstraction, agent D finding out new abstraction was not needed in the first place and cleaning it up. Management s ecstatic seeing 4 merged PRs – time to make a blog post about merges per day growing more than tenfold.
Because they are using 100x more tokens with the latest best models only.
If you cranked up turns and only used fable level model for every interaction, you would get way way less jank. Plus when you have essentially unlimited tokens, you can always do extra AI review passes to reduce jank.
I think for us, outside of Anthropic, it is hard to imagine the scale of AI agents that they are running internally . We are talking about probably 10s of millions $ worth of spend monthly if not more.
lot of people here are criticising the set of features stuck together or "janky crap"...I don't know where you work, but I have seen a lot of that in all my career, and still do... and it was done by humans. The difference is that ai is doing it faster and can also quickly change it in the future, so not really sure it's the same level of problems.
This is a big change in how we think about software development, bugs, and features tbh in terms of speed of reaction, identifying problems and caring about janky code or performance, because solving those problems, if they present themselves, is cheaper than before.
Honestly it does not matter provided u develop tests that show success. D u really care how its coded if it is amazingly fast efficient and solves the problem? Maybe in some projects but not all.
> This is my disconnect with this kind of speed, how are they avoiding the janky crap?
Because you are using LLMs to write code that you would be writing by hand.
Use it to write thing you wouldn't have time to write by hand and that are not mission critical: custom linters, support tools and so on. You know, each time you think to yourself "hey, I would write something to automate this chore, if only I could set aside one or two days".
You can also use it to document silly things you wouldn't take time to but that provide context to agents. Ask it to do a performance audit and to give you a few SQL queries and bash commands to gather stats about prod and logs. Then commit this document.
Will you read it? No. But your agent will use it to write code that's fit.
Keep doing this and 1/ you will be more productive, 2/ your agent will progressively improve so that you will let it do more and more.
> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
We still need the human creativity and spark in the loop to come up with these fresh, new, bold, creative ideas. I think if you let the current version of Claude run forever, it still wouldn't have the creative spark to create its own new source control system.
This is the future of a human software engineer's job.
Agree but every statement I now read from Anthropic just sounds like marketing bs to me. When they say 20x less and 4x faster they sound like Apple with no real data comparison or any detail at all
It's really incredible how they've squandered so much goodwill over the past few years.
While I think their models are great, and they deserve a lot of props for caring about safety (their interpretability work is great), their public tone just leaves so much to be desired.
Apparently they also don't understand the Streissand effect, given their recent blog posts about GLM 5.3.
> and they deserve a lot of props for caring about safety
At this point I believe they NEVER cared about safety. The CEO doomerism and the whole "slow down the development of AI" far cry is another marketing strategy based off fear
>We do 25 gazillion code checkouts per day and we merge 30 trillizillion code changes per day because we have 10 megazilion agents working on TEN MILLION projects at the same time in a 100 PETABYTE monorepo. Git is just not enough for us...
Every time any detail comes out of Anthropic or OpenAI it usually turns out whatever they’re doing is just incompetence. Right now I’d put money on that anything they were doing with git was incompetent garbage and they decided that the path of least resistance was to vibecode this thing.
You're talking about a company who sells software subscriptions of multiples of 1x, 5x, and 20x, without telling you the base amount of what "x" actually gets you, and changing it over time.
For the last year I keep reading from people and companies who say things like this, “I gave it one prompt, walked away for a while, and came back to something perfect and amazing.”
Meanwhile I’m here burning company tokens on top-tier models, and I still have to course-correct or problem-solve with my agents every 10 minutes…
English language needs a word to say “I don’t think you’re lying, but I also don’t believe you.”
The word is “bullshit.” Not as in “you’re purposefully making a fool of me” but as in “you’re not lying but your statements are uncorrelated with the truth.”
I've heard similar sentiment from a few people in my network. When I dig in, I find that these people are often working on much "simpler" problems. For example, they're building dev tooling (like testing frameworks) instead of working on business-facing services.
It's easy for them to one-shot solutions because the problems are very well defined, and the agents can easily reason about how the software will be used. The closer you get to the "real world," and the more complex the business logic gets, the worse the models get at making the right architectural or design decisions. If you're not watching closely to give input and steer the agent, it gets out of control pretty quick.
I find it ironic that these same individuals often loudly and publicly celebrate each new model release (as if they're early investors instead of early adopters), and tend to be very optimistic about agentic coding -- even though their jobs are the most at risk of being automated due to the simplicity of their work.
The closer you are to actual business outcomes and real-world software projects, the more you tend to understand that agentic coding patterns struggle to navigate the messiness of the real world. If you're working as a software engineer in an actual major industry (healthcare, insurance, defense, infrastructure, etc), you're probably going to be more skeptical when a gung-ho SDET makes bold claims about how LLMs are going to take everyone's job within the next 3-4 years.
> “I don’t think you’re lying, but I also don’t believe you.”
Have you seen Opus 5.5 code? I don't know any developers who'd even remotely match it at a similar breadth. In fact I haven't found any discipline yet where it did _not_ develop an elegant solution, on top of achieving the goal. Senior human developers might be better in one or two specific niches maybe, but how many projects nowadays are confined like that
Where exactly do you see human creativity and spark in the loop or fresh, new, bold, creative ideas in
> We asked it to rewrite the whole thing, and with no further instructions
They simply asked the model to rewrite the code without suggesting anything to the model which counters your creativity argument, which I see many people make too, but which I believe is not correct anymore. Models IME already are creative in ways I personally find it difficult to compete with. I still occasionally find something that the model in the first iteration would not to but the model is already far superior with its capabilities than I am.
Well, Anthropic used model A to output the version 1, and then used model B to rewrite the version 1 into version 2, so the handoff was machine to machine and not human to machine. I read that part of an article as if it was their internal assessment test of their new model B (Fable) capabilities so that they can see if the new model can find interesting things but by intentionally being vague and without implementing the bias into your prompt. I use this technique very often as well for two major reasons: (1) I don't know what I might be missing so I let the model do the unbiased exploration of problem space, (2) I am curious how far the given model has come with its capabilities. What Antrophic did in this case is an example of (1) and I wouldn't really call it creative since anyone could prompt the model with "rewrite this code".
That is not the quote parent comment was about wasn't it or am I missing something?
In case of your comment, I could have agreed to some degree but what you say is also not necessarily true since we don't actually know if the new source control is the result of human creative input or if it was also suggested by the model. Having worked with these models extensively I am inclined to believe that both can be true.
Well, the quote says that rhe first iteration was habd coded. So I assune that this means humans fully inchangé ofnthe décision-making; otherwise, the statement would be dishonest, wouldn't it?
Not to downplay Anthropic's engineering work here, but I suspect we'll see a lot more reinvention of the wheel, even at smaller startups, as AI makes writing code significantly cheaper.
The tradeoff between building something custom and adapting an existing solution is changing. Whether maintaining all that custom software turns out to be equally cheap is another question.
I think the article would tend to agree with you, given its position on the reluctance of open sourcing and/or sharing the actual Scion software stack.
There's so much intricacy and subtlety in each organization and even individuals' environment being incorporated into tooling that expecting these tools to generalize at this point is perhaps an expired ethos.
I even see this in the tooling that I run in my own personal stack. Many projects are candidates for sharing, and I have nothing against it, but they incorporate such specificity toward my own environment, including things like hardware, that creating shareable and more generalizable projects from them adds more to the maintenance surface than I would care to support.
Not a reply to TT, but I just saw your nick and thought I'd respond. We both commented on a thread recently, and your response was really insightful and just a pleasure to read; I didn't take the opportunity to respond there (locked). You should, like, write a blog, man. I infrequently wish HN had a "follow" feature, but this one of those times. Peace.
When people wonder about where all the productivity of LLMs is going to I think this is part of the answer.
There is no doubt everyone is far more productive with LLMs in generating code. However, a lot of productivity is going into things like that. Have you always been bothered by the limitations of Grafana? You can generate another dashboarding solution that works best for you. You know that deployment team that is so slow and annoying? Now you can generate your own deployment engine. Have you always hated how kubectl lets you manage one k8s cluster at a time? Now you can ask AI to generate fleetctl. You can start your project with DynamoDB, rewrite it to mysql, then to postgres, then to duckdb. Start with Python, switch to Golang, then switch to Rust. Deploy on AWS, then move to Azure, then move to a hybrid option then throw colocation in the mix. Don't like your source control? create a new one. Json is too limiting? Invent a hybrid JSON/HCL language. The browser sandbox is limited? Make a desktop app, make a language, make an OS, make a kernel.
> However, a lot of productivity is going into things like that. Have you always been bothered by the limitations of Grafana? You can generate another dashboarding solution that works best for you. [...] The browser sandbox is limited? Make a desktop app, make a language, make an OS, make a kernel.
This is one of the core reason that I dislike the LLM hyping. People are just reinventing stuff and when you ask them, it's actually a very subjective reason. Ok, anyone can do what they want with their time, the issue is that they want people to follow in their steps.
Not saying that you need a novel idea to be worthy, but those random projects have no guarantees behind them other than some cursory testing. All software have bugs, but I can't interact with something that is randomly built.
> I am one of those people that reinvents the wheels because external ones don't fit my use cases
That’s OK, but I’ve yet to see someone articulating what exactly doesn’t work in the old product. It’s always “Let’s rewrite or reinvent 90% of it, then add a minuscule variation”. The variation may be interesting, but the reinvent is what’s make it untouchable as no one (including the author) has time to ensure things are right.
A patch or a plugin is ok, but not a whole new project with dubious claims.
I agree. I think if everyone doing that is doing it on their computer like you're then it wouldn't be a problem.
When other people start doing that to other teams at work, other people in open source, other people in professional settings, etc it becomes a problem for various reasons. The idea that I'm gonna gift you with a 75k lines of python to "just run it" and ask your agent to hack on it might be fine (who cares as you put it). Now scale that to 47 of such projects (a month)
I think the point argued is that the hype tells us AI is worth a shit ton of money (see anthropic/openai/etc valuations) but that a lot of the benefit is in economically marginal use i.e. make some dev's life a little bit comfier.
Eh, I could be quite specific and detailed about the "things that don't work" that are motivating me. Because most of my prompts require actively thinking about at least one of them.
Assumptions. I've rebuilt a number of inference engines to run on windows and unify the dependency venv to last version to save space, unrolling model blocks so they fit in my 16gb card etc. A lot of thing thay I was locked out are possible now as i am no longer held back by invariants decided by someone else, and because I do not have to spend time to get things done. I've changed games to have a less grindy loop, added features, or making learning them easier. Ive fixed drivers, patched versions, brought dlss5 to game weren't supposed to have it
you know that you can reinvent the language that youre talking. but the value of common language is that you can use it to interact with other people. same with software its the synergy of common understanding.
You wont be able to do everything alone, and with llm you can push the barrier so far that eventually when you will need to communicate, no one will understand what youre talking about :)
There will be no common framework to build on. Like imagine talking now to someone from ancient rome, they will not have an understanding for what most of everyday words you are using today.
its not about translation. its about the concepts. there was no concept or microbe, elcticity, gravity integral, rational number etc. so imagine everyone now would have totally different concepts that only they understand. how do you communicate without common ground.
For some things, you'll need to build a working framework from grounds up. Electricity or more complex maths would be an example. For some, it might be enough to repurpose the existing framework they already have. For example, microbes: if you're trying to communicate something about health and hygiene, chances are, the other side already has a half-decent framework built around malevolent spirits or taint, that amounts to the same thing in practice.
Or, just tell the universal translator to ELI5 it to the other person :). Or "explain like they're a caveman". Our UTs are fully capable of translating concepts between conceptual frameworks.
There is surely a tremendous amount of waste and noise right now. But does it _actually_ matter what the code looks like for a desktop app that’s useful to a small handful of people? Every single implementation has at least one person who thinks it’s dog shit, no matter how correct or perfect it is. We have to get past this. If you really can’t stand it, build the thing yourself or some deterministic rails to reel it in.
And, there are also people who know what they’re doing building things they couldn’t before.
This situation is a net positive, and not too different than before. It’s up to you to navigate the noise.
> does it _actually_ matter what the code looks like for a desktop app that’s useful to a small handful of people
Not really. But equally, I imagine most people have a similar experience of software development to me - complex, messy and confusing code also often has poor performance and a high rate of bugs.
Sure there is, you said it yourself. You can make all those changes because it's easy, the lock-in effect is waning. We can move around and take better deals and choices.
This is actually a scary side of AI for providers: we use our own AI to research better choices and escape dark UI patterns and vendor lock-in. That information asymmetry and the difficulty of leaving a provider used to be their moat.
On a system level I think AI code is more like biological diversity, it hedges against monoculture risks. We don't all fall down at the same time if we don't replicate the same code.
I don’t really see what lock-in means for 90% of what AI is replacing. It’s more of a fear of commitment which isn’t exactly new. Enterprise space architects have designing systems to have pluggable everything from language, frameworks to carbon based life form.
As for the bio-diversity, I guess we’ll see the next time us-east-1 goes down.
It's surely subjective and changes project to project. I built a personal app on iOS with GPT last month because I didn't really like any of the flash card apps I tried for free. The way it works works very well for me and if I have any feature requests, I get to actually tell a thing to implement them, rather than sending a dev an email and hoping maybe they'll consider my idea.
At work I've been building an end to end system to do a certain process. The codebase is definitely more spaghetti than ideal and I really am concerned about maintenance down the road. But on the other hand I was able to POC it faster than anyone on the team would've before LLMs came along. We're already getting useful results from it too.
Your post above describes the value. All the annoying restrictions and UI/UX decisions made by SaaS providers that don't work for you. The value is being able to essentially wish into existence your ideal tooling, unburdened by the tradeoffs and decisions that some product designer thought was best for the masses
In a few days, I created a few apps with AI. One saves multiple mechanics 5-15 minutes per day and another can save multiple hours of mistakes during the project, and one saves a lead engineer 15 minutes per day. These savings should add up over multiple years assuming all keeps running as it does now. I’m having good hopes because there have been few bugs so far. Mostly just tweaking.
I’d never tell people what to do with money or free time. If someone wants to build many apps to help them personally, all power to them.
I can’t imagine a worse hell in a team or collaborative setting though. Imagine (or you don’t have to imagine since it’s already happening) asking a coworker “how they check how feature X is doing in prod” and their answer isn’t a query (regardless of how long it is) or some slightly complicated script to run. Instead their answer is a 27k line custom Rust desktop app. Then 2 days later you discover that the big green “All Systems Operational” is just a bit of static text, and in fact not all systems are operational. The app just doesn’t check things correctly.
Originally I felt that there was going to be a reinvention of the wheel, until I saw the "shape" of software development changing, and the tooling needing to support the new flexibilities in positions and process.
Software development is not as much de-risking at the front like it used to be before building.
Now teams can build lots of stuff that works great, except it's the wrong things.
Most things are becoming the understand the problem "shape".
Understanding the shape of the problem isn't something LLMs can always make as.. svelte as early on.
> Now teams can build lots of stuff that works great, except it's the wrong things.
That used to be called "time wasting". If you want to build software that will be used, you need to interview the users. Maybe a few things does require research to nail things right. But most software are user-facing, the research side is about optimization.
> Most things are becoming the understand the problem "shape".
Then should we be so reluctant about "trialing" invented solutions. It's the same thing with most AI projects, they're trying to justify the project after the facts while being very reluctant to get user insights.
Time wasting is a definite concern - especially when vibe coding doesn't see the impeding brick wall(s) it's about to hit.
Vibe coding can be kind of a new type of cowboy coding. It could can be harnessed productively if done mindfully.
Interviewing users is critical, more importantly solving problems worth that people are lining up to pay to have solved before writing a line of code is critical.
Trialing solutions is always worth learning from, I usually ask about the problem that lead to building it. Sometimes that can be too narrow or vague of a story and you can learn how much of it is a solution looking for a problem, or vice versa.
We're trending towards just building the thing fast and getting feedback sooner on it. It's interesting to see end users actually trial a UI very early on instead of just being shown designs.
> If you want to build software that will be used, you need to interview the users.
Some people write (I hate the word "build" here) software for non-business purposes. As nice as it would be to interview potential users, there's no clear way to discover and contact them.
In the educational universe, this "interviewing" is more common than not. Teachers are asked to give feedback on textbooks, calculators, and software, including lots of educational half-baked A. I. garbage. Some even offer us food for our feedback. Can you give an example of a type of software which is developed with "no clear way" to contact users?
> Understanding the shape of the problem isn't something LLMs can always make as.. svelte as early on.
My suspicion is that this simply won't go away as the models get better, because it isn't a question of how good the models are. It's a question of the human operators knowing exactly what they want. Which is, I conjecture, usually impossible until you try something and critique it properly.
This was always a big problem even in the pre-LLM days. The difference was that you only needed to be better than your competitors at solving the customers' problems. There's a reason everyone gripes about the software that they use and it's because the software is built to satisfy a very broad customer base, broad enough to justify the engineering time put into it. Now you're competing against tens (maybe hundreds) of other firms all building for the same competitors so there's a lot of pressure to get the shape exactly right.
There's a reason the product-minded highly-technical engineer was always highly sought out.
Recently a colleague of mine shown us a demo using a fresh model designing a building construction detail live, in minutes, which is a function that we struggled to bring into our construction design software due to resource limitations (and legacy barriers) for long. All this in the context of offsite constructions where the design can directly be exported to the right CNC machines in the factory, and arranged for delivery and assembly order. We have let ourself amazed, despite the controlled demo situation and the small detail, it was a remarkable performance. For a minute or so, but thinking further, why would the customers (construction design engineers) of our competitors flow to our legacy software full of legacy pitfalls they know about and have to live with when they can turn to AI for a fresh design solution themselves, just like we did? They know the context better than us anyway, just do not have time for bit-carving. Why would we have customers when customers of our customers can turn to an AI for a design that is buildable? I was consulting one model about a structural engineering detail, and it was quite well informed I have to say. From training data alone. I felt, maybe I need to rethink the rest of my career, that is not that long but still uncomfortably long for being predictable beyond a few years.
Well, things are not that bleak, probably, at least not for our customers. Regulations (accountability) and physical world matters still require them. But not us for our customers, not that much and not for too long. They tend to be the tech savvy folks eager to jump at adjacent domains when it saves them a percentage or two off the construction costs (not a small pile of money), and AI with its programming language of English and deep knowledge pool is not that big a jump from their standpoint. Our domain is pretty well formulated, regulated, documented, inherently built on do-check-adjust loops already. With a thing like AI it might worth to try making and improving a design solution in a second loop. But they also started to invest in robots already driven by software for their factories, AI might jump in there, and I wonder how much more until the regulations are forced to change by Super Intelligent (R) politicians for the sake of AI accelerated built environments (housing crisis, you know!), shifting the AI use further out towards generic population, wanting to order a new two bathroom condo for next Friday from Alibaba. The conservative construction business is a nuisance and obstacle in the way of modernisations anyway, go around it!
I will go to work tomorrow, and for some time, but the picture above feel less and less fictious with each passing day of AI news and usage.
The question was never whether you could do this or not. It was always whether it was wise to do so or not. More than one company has gone under writing their own editor, compiler, database engine, operating system, source code control system, ticketing system, CMS or ERP.
Building something custom often is re-inventing the square wheel. I'm dreading the idea of switching jobs only to find the LLM generated software equivalent of Terry Pratchett's “Ordinary madness he could deal with ... But Vorbis had passed right through that red barrier and had built some kind of logical structure on the other side. Rational thoughts made out of insane components.” from Small Gods.
This was already a problem in the industry, many companies reinventing the wheel on larger or smaller scales; with AI it's becoming easier to do so. Ultimately it's fine as anything not successful will just die out again.
But also, some of these internally developed tools get published and end up pretty successful.
Previously there were two problems with NIH solutions:
- they have implementation cost.
- they are usually born from arrogance mixed with lack of understanding of existing solutions and their tradeoffs, and typically end up being a net negative.
Before LLMs, the cost is what primarily used to stop us from building them, it used to be a natural filter for bad ideas. Now that the cost is gone, it's nearly impossible for most teams to stay disciplined and not be building those.
Why wouldn't you build a software factory? If you succeed, the ROI is infinite. Who cares about immediate customer needs, we're building a utopia here. Once it's done, all customer needs will be solved. Just one more month, I promise.
> In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day.
> The history of our main branch alone is almost 100 GB
What value do you get from carrying around all of that history? Your code is changing so rapidly that it looks absolutely nothing like it did a year ago, so at what point are you going to want to see what it looked like back then and every single change that happened to it since? Wouldn't you benefit more from creating a tool that can semantically truncate history entries that are no longer relevant, since even with scn your history will continue to grow out of control?
I think your lawyers might have a word with you, too. Source code history is more than just a record for developers. It's also a historic record that can become relevant in legal disputes.
While true, if the 100GB of history that isn't important anymore is dragging you down, create a new repo with today's code and move from there, then archive the old history for e.g. legal reasons.
Legal argument is already a strong one, but I feel that the training data in there must be invaluable. Such a mass of shipped features, regressions, refactors, complete rewrites and pivots, all with full internal history, is a goldmine. Whether archiving and switching branch might be in order, that's a fair discussion though.
Every now and then I like to browse the claude code release notes just to get a sense of the number of changes in that product over time. There's little choice but to simply count bullet points if you want to try to quantify it, but it's interesting how that count over time compares with the charts in this article:
I think one thing is consistent across all of these visualizations: they moved the needle. Hard to say what that needle points at or where the redline is, but fleet change is afoot.
> Very often, we found that the role of the engineers was to tell Claude “no.”
That line stuck with me. I wonder how the engineers building Scion feel about that? If that really is the future, then I guess I have to find another profession. Retirement is still too far away.
I have found this to be true as well with modern claude, and to some extent codex. The blog summarized it perfectly — perhaps as a result of so much RL on security/adversarial review, the models currently will speculate a (valid, but unlikely/out of our design space/deep edge case) problem, and then cheerfully triple the codebase size and cut speed by 10x in order to fix it.
It feels exactly like talking to an experienced security engineer suddenly tasked with designing a large system — right now the tradeoffs that an architect makes don’t tend to be there, so quality oversight gives much better outcomes.
This article doesn't do a very good job of convincing me that git is technically incapable of handling the described workload. There are a lot of steps between vanilla git and building your own VCS from zero.
If I was an investor I would be on the phone with Dario asking why we are wasting engineering talent on reinventing source control and churning the product with 30k pulls per day.
>If I was an investor I would be on the phone with Dario asking why we are wasting engineering talent on reinventing source control and churning the product with 30k pulls per day.
Indeed, while struggling with VCS is a right of passage for any tech company, I'm not sure why what they described couldn't have been built as a diff/patch API in front of git.
This is why you are not an investor. (Cue McMahon meme), A marketing campaign to get eye balls across all the major tech outlets and take up headspace in everyone's trending feed? Oh yes. A potential, closed source alternative to github you can slowly wrangle your users to move to? Hell yeah. An always online centralized design requiring users to stay connected with the server for every operation? Let's go!
They had a CI/CD problem, so they fixed... version control.
They've ripped out and replaced the most carefully thought-out part of the CI/CD chain with something less carefully thought-out, while leaving the teetering house of cards in place.
Yes. In a world of bicycles[0] and horse carts, they realized neither are sufficient for their needs of cheaply and quickly hauling heavy loads across long distances. Instead of leveraging "the most carefully thought-out part" of using bicycles, they slapped a steam engine onto a cart and fastened it with some bicycle part, and guess what, it works, and hauls more stuff faster and cheaper than either bicycle or horse carts ever could.
--
[0] - If you can even call anything built before modern petrochemical industry a "bicycle".
Their whole point is that none of the existing wheels can actually handle the load they need.
I don't know why is this even controversial. Agentic swarms break scaling assumptions all across the typical software development life cycle; VC and CI/CD are just few obvious cases.
7 engineers is hardly a massive resource allocation. Even if they produce nothing of value the lessons learned along the way will be valuable - They're essentially dogfooding agentic swarm development which will help inform their future product direction
I am about 80% certain that using well configured Perforce could work just as well while saving Anthropic some money, considering the salary of Anthropic engineers.
Anthropic is at its core an R&D company; they invent things and sell them. If they invented a new version contol system they can sell, or if they solved a bottleneck that prevented the ability to develop stuff faster and better, it's a worthwhile investment.
The cost of innovation or "just" writing a new tool is plummeting fast.
What is are the benefits of supporting extremely high churn rate in version control? What is the code doing? What problems are being solved? Are changing requirements the driving factor, and if so, why are requirement changes accelerating?
> ..., the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.
If the goal is to write software which solves real problems, how can the demand for code grow infinitely?
I find the accelerationist mindset utterly ridiculous. All that busywork and for what exactly? What effect has all this overproduction of LOC had on the physical world and on our existence (beyond the externality of erecting huge data centers in every corner of the globe and the consequent ecological impact)? Is software getting better (in terms of quality, security, efficiency)? Are we getting better government services? Are we getting better food? A better environment (silly question)? Are we working less, as tech titans keep promising us? Are we getting anything better? Not to mention all the negative aspects of this technology and the financial apparatus driving it.
I agree with the sentiment, and the answer to all these questions is usually ‘just wait 6 more months’. Like what is tangibly, actually better in the real world right now with AI? Whenever I ask this question people pipe up with a side project or a new SaaS, but that stuff is just not very important compared to the things you called out. Food, services, QOL.
It seems to me like the AI labs are just pushing the envelope with blind faith that”something” will happen if they just get enough development speed. Nobody seems to have a coherent idea of what that something is.
Seems to me the goal is to create a convoluted spaghetti mess in every code base so that humans no longer have a chance to understand how it works and thereby force all software development to go through LLM obfuscator-deobfuscator cycles.
> Like what is tangibly, actually better in the real world right now with AI?
I took a photo of a tree with chatgpt recently and asked it what species it was. That was kinda neat I guess. Someone forked a ps4 emulator with a bunch of Bloodborne specific improvements so its more playable on my setup. It's kind of okay for medical stuff.
> I find the accelerationist mindset utterly ridiculous. All that busywork and for what exactly? What effect has all this overproduction of LOC
Yeah I wish this whole energy was towards optimization and improving hardware.
One thing that people are finding hard to believe is that the frontier labs are cultists who want to "create god". Hence the focus on improving intelligence so they can get to recursive AI, as well as the fear-mongering.
Aanthropic-openAI duo cannot go bankrupt soon enough. This industry is going to flourish with those two out of the picture. They're the pets.com of 1999 trying to have a say how the internet should be.
All that busy work was for going from zero revenue to $50B in revenue in about 3 years, and releasing AI systems that are getting better and better at a ridiculous rate. AI isn't solving all the world's problems but you can't deny that they are extremely successful with this method of operating.
It's a fair question to ask. If literally all the economic growth (and then some?) is going to AI companies, then nothing else is improving, in the aggregate.
The broader question is definitely relevant, I was thinking more in context of the original post justifying their need to re-invent source control.
I definitely think that's an interesting question though. I know that in the physical product development world I'm seeing really odd asymmetric speed ups. Creating simulations is essentially automated at this point, stuff that would take days to code by hand can be done in an hour. Firmware debugging feels 3x faster than before. But the big slow stuff like manufacturing, assembly, and testing are completely unchanged. All of that little stuff probably adds up to maybe a 5% faster product development cycle than before if you're really leaning into AI. It's the type of thing that will compound over time especially if it's happening all over the economy, but won't blow your socks off.
I think part of the concern is around “could it be better”? Like if an elite athlete eats junk food all the time. Sure it’s working, but is it because of the candy?
Anthropic is often criticized for their poor UI/UX. The models are undeniably powerful but the packaging is typically poor.
This is not to drift into the conversation of profitability either, to be clear.
But the revenue is just because they’re selling at a huge loss and are being allowed to do so because lots of stupid investors are betting that one day they won’t be selling at a loss. That’s not economic growth. Anyone can have revenue selling diamonds for a penny
I think their success is pretty debatable. This is a company that lost 8 billion dollars last year, is in an extremely expensive race with other companies, and is basically the premium version of something that's now becoming a commodity. They might be doing some impressive work but as a company their outlook seems questionable.
Since when have we expected companies scaling that fast to make a profit? The fact that they've had profitable quarters already is pretty insane.
There are definitely risks to their business long term from commoditization and a lack of a moat, but I don't think that detracts from my original point that they are building something highly desirable with this new version control system today. They are probably the fastest growing company of all time in terms of revenue and valuation? They're not just slopping out more and more LOC for no reason, which was the original point being discussed in this thread.
To play the devils advocate here (just for software), I would say principally this:
"A world where starting is incredibly easy and finishing is more complicated than ever"
LLM's do allow you to oneshot huge starting bits of work in record short time. The frontier of the amount of work they can oneshot is clearly getting larger and larger. But I am still not convinced that they allow you to maintain existing work at a cost that is below a human being.
But, if the frontier of what you can oneshot continues to grow unbounded, then it really will not matter. We will eventually oneshot all software and maintenance will be a thing of the past.
The funny part is, no one is making any money off this either. Traditionally we’d say that profitability is not necessarily an indicator of real value for humanity, but with LLMs we don’t even have profitability! The only people making more money here are manufacturers like Nvidia and data centre companies
What defines a "real problem" but that someone is willing to pay money for it?
Who decides what the "real problems" are?
Imo most of the economy and wealth creation is around activity that is not solving a "real problem", but who am I to say if someone else is willing to pay for it? Presumably, for said person, the problem is "real" enough.
Yikes. So much evil done in the name of God. I would think nothing would be further from the conduct of a "moral actor" than equating themselves with God.
Completely unnecessary. Secular humanism for me, thanks.
If you are constantly churning your codebase to the point that you need to invent a system that handles your churn... then you must at least be admitting that the previous 6 refactors you wrote were not good, but NOW you have it right.
I feel that good, well designed code change by accretion, not by rewrites. Code that remains stable for a long period of time was likely better than code that needs be continuously rewritten (at least with humans writers).
But maybe this is just a human-centric view of what good code means? That LLM "slop it until it works" approach to development will have a different measure of what quality looks like?
idk, over the past 2 weekends i have a repo for hacking on fujifilm camera firmware that's gone from zero to tens of gigabytes: i've got an emulator for older arm cpus like the one in the X70 (2016), a Hypervisor.framework VM for running their newest AArch64 software for X100VI (2024), a complete C re-implementation of the x70 source, many tools for injecting code and executing on live camera RAM over USB. the emulator has gone through 3 rewrites: first it was python and naive, then it was python with hot parts in rust with a more mature device interface, and now it's rust with some cold or experimental devices still python.
i really don't want to brick my camera, so the deterministic emulator stack lets me do integration tests end-to-end. if i was doing this by hand with no ai, i'd probably have to run stuff on the camera just to test it. but because ai, i get deterministic simulation testing, at the cost of needing to burn $100 of subscription on it.
ai code is more like clay you gotta continuously keep shaping over time, but you can also get very very far, very quickly if you decide the code itself doesn't matter and all you care about is results.
Agreed, a buddy and I wrote a simple mod that shimed into Warhammer 40K: Rogue Trader and were able to pull all of the art and description assets for that game by walking the object graph. Took about 4 hours total (including the site).
Similar to your camera, we would never have been arsed with doing that much work were it not for LLM vibbing. It allowed us to get something working (sufficiently) very quickly.
At what point of maintaining this code would it be cheaper for me to actually find underlying patterns and program this myself with some actual thought, rather than continuously "shaping the clay" of my LLM slop? These questions keep me awake at night (not really), and probably predict the future of software engineering!
I'm curious, was there a specific reason you went with that solution for grabbing the games assets?
Because asset-ripping is a pretty widespread, well documented practice in gaming - it's how a lot of modding communities figure out and expand on the games they mod.
A quick search shows WH40K: Rogue Trader is built on Unity too, so any number of existing Unity-specific asset-ripping tools should have worked for your purposes. Though I should say I've never actually dabbled much with modding Rogue Trader, so maybe I'm wrong?
It was January I think and I don't remember our thought process. I think we mostly just remember looking at the fandom.com or whatever crap was there and thinking how out of date that stuff was.
We also just wanted to try some modestly sized project with an LLM. Neither of us had any real experience doing any of the things we ended up doing for that project (other than my making the website and him understanding the graphics pipeline).
So, unfortunately, no, there was no specific reason I suspect.
I think that they're essentially solving the wrong problem.
Imagine that you have git but you don't even have a concept of programming language more complex than assembly. All your commits and git history are pure assembly. Everything is cool, until one day C++20 randomly spawns into existence, and a bunch of guys decide to use it in the company. The issue is, the compiler generates so much assembly, that your existing workflow cannot keep up with it. Your assembly git repo grows into hundreds of gigabytes, with PRs that are pretty much complete rewrites coming every single day and there's no way to review any of that shit.
What Anthropic is doing is redoing git from scratch so that it can handle all that assembly, while the correct thing to do is to commit C++ code into the repo, with deep trust that you'll never hit a bug in the compiler - basically impossible for initial versions of compilers, taken for granted nowadays.
What I'm saying is, Anthropic should create a versioning system for prompts, because they are the true "source code". Given the same prompt and the same sequence of random numbers, LLM will generate exactly the same code, which sounds awfully close to how compilers work. Once in a while put the generated code as a point of reference+ caching and presto.
Except they don’t right? They aren’t deterministic. All the hypemaxxers over at twitter keep saying you need a way for the agent to verify the result itself. Would that be another prompt then?
No one is actually one-shotting new code on top of the existing repo and just checking it in. Even in a simplistic case, it’s multiturn. You give it a prompt, and it writes some code, tries to build it, and it doesn’t work. So then it keeps iterating, and changes some more code until it builds and then does the same thing again until it passes tests. And if you have a complex system, it might do a lot more CI.
The input to this is not just the preexisting code, the prompt you provide, and some random seeds. It’s also the output from every single tool invoked, which means you need to record the state of every single tool and every file that was read, every API that was called. It means that you can’t upgrade Claude. It means that you can’t upgrade the compiler. If you change anything in the state of the world you no longer have replicability.
Aside from all that, I don’t know why you would want this anyway. Storing a bunch of prompts doesn’t seem to actually gain anything even if it worked.
> What I'm saying is, Anthropic should create a versioning system for prompts, because they are the true "source code". Given the same prompt and the same sequence of random numbers, LLM will generate exactly the same code, which sounds awfully close to how compilers work. Once in a while put the generated code as a point of reference+ caching and presto.
Bad analogy, cause this is not how tooling works. Tools beget more tools - there's value in specifying "these are the sets of tools we need to make", but unlocking new capabilities doesn't happen unless you... make the tools. And use them. And keep them stable. And watch them break, or become obsolete, and build on top of that. The world has state and path dependent.
My thought too. Like, I get that Claude gets new features pretty frequently but.. what on earth are they possibly doing that needs that much churn?
The only thing I can think is that they're doing a google style mono-repo, which like, fine, that's a thing to do, but I really have trouble picturing that changing code at that rate is a useful thing to do.
The churn is rewriting 50+ line comments on every function. These comments will also include every place that references this function, as well as random historical data.
Simple, they want Recursive Self Improvement and worry that git will be the bottleneck. So they are making source control as efficient as possible in preparation for that day.
From the first paragraph: "Claude is able to autonomously iterate on research loops, micro-optimize complex programs, and build new services from the ground up at a speed and scale we haven’t seen before. The bottleneck, now, is all of the developer infrastructure built for human scale: source control, builds, and CI."
As I can see it is natural way that big tech will concentrate their source control into powerful server side, Facebook did that (their monorepo use a customized mercury), and now Anthropic. But I believe we should remember what git has changed is not the way we commit code, but more the way we share code and open our code. Before git, there is cvs/svn and it has never given us the real distribution system as Linus ever talked. After git, besides the speed, the most important thing as I believe is that any one can own the full history of a repo, and that's the foundation to true open source. Anthropic now? a backward step to a closed garden that's all, and they have all the tokens, so let them be happy with themselves, that's all.
I came into this blog post a hater, but honestly - hats off to the team. This sounds like it was a very arduous project, and I don't envy the engineers that have to sweat over handling Anthropic's scaling commit volume.
To that end though, this line feels like it (intentionally?) undersells the engineering team -
> Very often, we found that the role of the engineers was to tell Claude “no.”
To me, this blog post highlighted the immense value that domain-specific knowledge still carries in engineering. I wouldn't be surprised if their team was comprised of former Facebook/Google/Github source control engineers. Individuals with real experience on these types of systems.
The dream for a lot of folks in the industry right now is that you could just staff a bunch of generalists on a problem like this, and through the power of Claude come up with a sufficient solution. I think the reality is that without the domain expertise to direct Claude, you would end up with something vastly different, and not really usable at Anthropic's scale.
I genuinely don't understand the core value they're pitching here. The tech is interesting, and I'm not trying to under cut that, but I question the idea that this is directionally valuable advice to nearly any company.
This amount of churn feels like a thing only possible with no real profit motives and a lot of money to waste.
Lots of activity is not necessarily synonymous with productivity, and it carries real cost.
So I think the design here is interesting, and the concepts seem applicable to where some agentic development may end up, but I question the underlying principles that drive this being the right solution space at all. At least if the goal is a profitable company.
I think its pretty straightforward no? If agents are doing all of the work, they'd probably need to not have bottlenecks. This is going to be any company in the future, not just Anthropic. Wouldn't you agree.?
30000 a day seems already beyond absurd, it's one merge every 2.9s. I'm sure they will soon have more than one merge per second. From the graph, the rate was slightly lower but already insane as fuck the whole summer. An just regularly high to insane high sooner this year.
Yet IMO the admin controls Claude provides just started to be adequate for a small company a few weeks ago (and even so are still ridiculously fragmented and seem fragily designed and not exhaustively documented), so I can't even imagine for a big company.
Do you need that rate of change to make basic controls? I'm not sure. Do we see such dramatic fast evolution in the products? Not really. Are bugs we report actually fixed? I tried once and it just auto closed.
Anthropic should optimise for something more interesting than number of merges per second.
Hysteria doesn't need to make sense. I think that's what OP is suggesting.
In the early/mid 2010s there was a similar hysteria about "deployments per day/week" metric. People were comparing products or teams by saying "My team deploys twice a week", "oh, yeah? we deploy 3 times a day". People viewed the north star of the CD part in CI/CD as pushing to production. The concept was simple, you push your code to main. CIs start churning, deploying it to test environments, then promoting it to staging, the pre-prod, then into blue/green prod deployment, then into prod. All automated, all touchless, happening on every merge, 5-10 times a day. That's engineering efficiency.
Many tried to follow that. It works great if your project is a Jekyll blog of course.
That’s exactly my question reading this. Granted I don’t know much about the training side ofbLLMs, so maybe there’s more code changes involved there than I expect, but in terms of public-facing software, they make, what, a couple chat apps, a product website, and an API?
Ar work, I'm working on some entreprise software which is 50x more complex than anything shipped by Anthropic and we have 2 orders of magnitude less code merged per day.
I wouldn't be surprised if a good portion of these PRs are multiple agents stuck in a loop constantly overwriting each other changes, with no human involvement at all.
great article. Besides the source control stuff, I found this very interesting (and matches my experience with agentic coding in general, now we'll see until hiring pipelines will take this kind of skillset into account). I am not sure however how easy it is to review 30k PRs/day even if only for intent, and how one can easily notice if the model is taking shortcuts on implementing said intent without reading the code...
---
Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. This was particularly common in our migration. For example, there were multiple systems that tried to brute force looking up when a change merged to main by linearly reading the commit log instead of asking our code review tool. Anthropic also makes extensive use of adversarial review, and, while this is incredibly effective in many ways, it can also be myopic, so skilled engineers needed to steer Claude away from over-engineering parts of the system in response to hypothetical concerns. For example, one such review was overly defensive about trusting another internal service, which resulted in Claude adding excessive validation and complexity. Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.
This is the single biggest skill of engineering nowadays. Rapidly looking at a solution and saying, "yep that's good keep going" or "nope, that's bad fix this" or even "wrong approach, do X instead"
> I am not sure however how easy it is to review 30k PRs/day even if only for intent, and how one can easily notice if the model is taking shortcuts on implementing said intent without reading the code...
Speed reading code is huge. You just need to focus on certain hotspots and know which things you're scanning for. The traditional approach of reading the code and trying to compile it or reason with it in your head is too slow.
I guess this is also where an adversarial model review of "does this code do what the PR comment says it does" is useful, agree on speed reading being a valuable skill for sure.
I do think that the lgtm gut feel looking at a PR requires quite some years of writing code and reason about it, so not sure how a junior can get that intuitive understanding if they start with agentic coding however.
With 1000 engineers and most PRs bein housekeeping we’re down to 10 prs per day per engineer. And as they said - this is engineers core job now. It’s a new kind of job.
Personally I can easily imagine one or two orders of magnitude more than that. They crossed the rubicon of not gating PRs on tests. In that land you can just roll out changes to production, a small subset of it, during code review to assess behavior. Or during key moments of coding.
I’ve been thinking a lot about this recently: what can we get if every change goes to production in 100ms, and what do we need to do to maintain stability?
> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
I feel like the advantage of having these models in house can't be understated. Companies with access to inferior models, trying to compete, will be left in the dust when it comes to general development.
We ran the repo ingestion in a loop for several months while we were split between Scion and git to keep things in sync, so this perf win was actually impactful for us!
Are you already operating with multiple levels of orchestrators and agents? i.e. one human talks to one or more orchestrators who then talks to orchestrators who handles teams of agents?
Yes, this is a common pattern. There is still an opportunity to improve how these agents write code without needlessly conflicting between themselves and we are exploring some ideas.
Others can confirm, but I suspect a lot of inefficiencies in large company programming projects are due to multiple people working on the same code - where you ultimately have to let both above and below average programmers work, and fixing bugs or improving existing implementations is more difficult because of company politics or interactions with others.
There actually was lot more hand-writing of the code back then! Or, at least, only moderately assisted by Claude with a lot of human-still-typing-things-in-an-editor. The ability to delegate larger chunks of code to Claude had a noticeable, step function increase with Mythos Preview.
The question of what to build rather than how is still central to development. If the article does point to success it's not as much a technical one but of product. That said I'd take these articles with a lot of grains of salt, knowing first-person how corporations spin their successes, including internally.
That's been my thinking lately. The SOTA models are good enough that they "solve" coding but still need directing. The product is still important. Products are human too, and need human attention (still).
Maybe that'll change in the future, but while AI can make awesome images without a good "director" they're pretty cookie-cutter and bland.
Yes, I think we'll find that technical limitations were never the bottleneck. I work for a large corporation and that has been the case here so far. Yes, a lot of things are faster, but issues with product vision, roadmap and cross-team friction aren't gone. They are actually maybe even harder to get right.
Yep, agreed. Like seemingly is the general opinion, taste and direction I think still matter a lot. Efficiency in it of itself is kind of ”solved”, but effectiveness, doing the right things, is not.
This is true for consumer software. I would imagine for developer tooling and especially internal tools these metrics are rather different?
A 10 minute CI pipeline is great and fine, but everyone absolutely cares if you can bring it down to 30 seconds. This software, by its very nature, is measured by speed.
I used to think this as well, but until it turns out to be a problem for the business worth solving (eg impacting $ machine), it will not get solved. There has to be a why now, otherwise plenty of "problems" get shoved under rug.
Internal tools, IME, are almost always short budget. unless it is in path of revenue, then there is $ and performance matters a great deal *if* more performance means more $.
The Scion project seems like a great example of identifying and addressing scaling issues (just in time, it seems). Just 7 engineers! It makes me think I'd pay Anthropic $300K/year to work on such problems.
But it's a sad example, for open-source and for small engineering companies. Instead of infrastructure being more easily shared with the collective, or valued enough to build a business, now companies can tailor even major infrastructure to their own needs, and avoid all that coordination.
I'm glad they publish their notes to encourage others, but I suspect we're entering an epoch where confidentiality matters most.
Anthropic's problem was merging code at scale, but the scale problem I have with SCM is really merging all the related docs; these get lost in a stream of a forest of comments, PR's, and documentation, all of which (like a wiki) goes rapidly out of date before anyone notices because none of it is automatically correlated.
> Instead of infrastructure being more easily shared with the collective, or valued enough to build a business, now companies can tailor even major infrastructure to their own needs, and avoid all that coordination.
Large companies have always done this. Tons of early 2010s open source projects came out of reimplementing white papers from closed-source Google projects.
It’s also the major downside of this kind of thing. Once someone gets acclimated to how things are done at a large company with custom tools, it’s not always easy to switch to another process, especially a simpler one. There are tons of stories about FAANG engineers joining a startup and being useless.
There’s real value in having consistent tools across an industry. It makes hiring and training easier. Even if you’re convinced that a LLM could crank out a payroll or ordering system that’s perfect for your org, you still have to train the accountants and admin assistants to use it. And you’ve now become the customer support.
The bar is lower with version control, but it’s still more friction than git + GitHub + a well-known workflow.
Excuse me for throwing up this stupid idea, but if code changes per unit of time keep growing at this rate and are not slowing down with infinite merging demand, it seems that this repo will be in a constant state of continuous change and we constantly roll forward with automated code and obviously no direct human supervision, then why even have source control? Why not just let software be fluid and, like Heraclitus river, never step twice in the same repo.
Technically, I guess you could say that a pull request is not a part of a version control system. GitHub has pull requests, but they're not a primitive in git, they're built on top of branches, commits, and merge/rebase. That said, I would expect a version control system built around agentic workflows to have something like it as a core feature.
Agents use source control, too. It's a pretty important part of the development process to understand how code has changed.
For example, you can imagine, at the extreme, that agents could get caught in code change loops: a code change seems appropriate, it is implemented, it turns out there's some external factor which makes it not appropriate, the change is reverted, then that same code change seems appropriate again, etc.
Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
In theory, an agent could use git-bisect to identify when a bug was introduced. But the operator would have to be more interested in understanding the root cause than in fixing the current form of the bug...
For me, Claude regularly looks at git blame after identifying a bug, then reads the commit message and other changes in the commit to understand the context of why that behavior was introduced like that. I wouldn't be surprised if it pulled out a git bisect
Then again, my standard instruction is something like "{bug description}. Find the root cause and propose possible solutions" so I guess I fit your profile
not really an option when you're working on a large codebase where a bisect means that a lot of steps could be multiple-hours of compiles. E.g if on a simple 12-step bisect you get 6/8 with a full rebuild you've already spent an entire work day on this
This is true, but at 30,000 merges a day we're talking about nearly 1 million merges a month, and purely autonomous agentic merges tend to be noisy. Is this still a tractable problem, even for an agent? Do revisions older than a few days, other than perhaps release snapshots, have any relevance for such a process?
Not all merges are equal, even if it's a monorepo, it's probably rare that you'd have to look at the whole codebase to implement a new feature or find the root cause of a bug. Once you start looking at the history of a specific folder, it becomes more tractable.
Of course that does require some folder structure, and agents are not necessarily great at that, but they're still decent enough that you do end up with many folders not being impacted hour or even daily.
> Agents use source control, too. It's a pretty important part of the development process to understand how code has changed.
This is a great idea, now they just have to use/do it. Maybe they can use it effectively by cloning the repo and having all the history at hand, rather than continuously scraping web pages with git content formatted for human brwosing.
I’m not advocating for no source control, but you can avoid loops like this with Architectural Decision Records. If you’re real lazy, let the agent maintain the ADRs and they will usually call out when these “painted into a corner” scenarios happen and you can then reason them around it.
Don’t let adversarial agents work on your code?? What is protecting an ADR against adversarial agents going to solve if they can just embed a highly advanced, obfuscated ‘rm -rf windows32’ anyway
> Hmm, I have destroyed the file again, that's odd. Let me rebuild it from a half recollection of what it was supposed to do based on my compacted context.
It's the right approach if your business is selling tokens and the customer doesn't know any better.
Anthropic can just replace pesky source control with a much better source guardrails. Aren't they all the rage and next step evolution in the computer security? Just ask Claud "pretty please do version control" and that should be enough, right?:)
You absolutely know there is one of those AI bros out there in social media posting on how they feel euphoric because their agents don't speak git anymore and just pass a zip with the source code around to other agents as handoffs.
>Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
most of the harnesses are checkpointing most of the data of a git commit aside from the formal release and messaging, anyway; thats why they all have a similar problem with SQL dbs exploding in size after a bit of time.
Right now we're in this weird transition where most of the harness systems are layering git tool use w/ the constant snapshotting AND a memory subsystem often, probably leaves a lot on the table in saving wasted token use to just ditch the vcs all together and rely on the already persistent and existent checkpoints and memories for reconstruction.
> Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
Or code comments which they are already doing a lot of. I used to get annoyed by their verbose comments but at this point I've seen them help a few times and it's not like I'm actually maintaining the code myself anymore so it doesn't really bother me
With the way they are shipping code, source control history has no practical value. If barely any code older than 5 days still exists in the codebase, and nobody reads the commit logs, you can throw out most things in a source control system. It is basically a database of latest version of code, that's about it.
I’ve been building and playing with this idea for a while. Mostly with simple chips and simple programs. It seems like a good idea on the surface, but unless they build the whole OS the ordeal gets much more difficult. There is also a lot of work building / finding training data without decompiling and labelling current exes.
Can confirm: In the 1970s our high school had a minicomputer for teaching programming, and it was turned off when not in use. After turning it on, we would enter the magic three binary instructions into memory using sixteen toggles on the front panel.
Give it six months and there'll be have Optimus robots toggling switches to load terrabytes of binary programs directly into memory. Nobody wrote it, nobody will read it, but it will run everything.
The point is that if we can put binary directly into RAM, then what is assembly if not a programming language? Assembly is source code in a programming language that is turned into binary by a compiler called an assembler.
If you don’t feel that toggling the binary 16-bit words into RAM demonstrates that binary is what actually runs on the hardware, and that assembly is source code for a binary, well ok. I find it persuasive.
I think we can take this further, let's just end the human race as we know since we've obviously just successfully completed our technology tree. There is nothing left for us to do anymore! It was fun guys. Well, it really wasn't. But we made a lot of money! OK, maybe not everyone. But some people did! And that's all that matters. Top of the leaderboard!
I believe you're mostly thinking of version history though. There’s plenty of other reasons to have source control we still need, like guarding against racing writes or continuous integration to ensure correctness. Even some industries require version history for compliance reasons.
I do love how AI is challenging us to rethink our tools though rather than blindly reaching to git for every project.
I don't think this is stupid, and I think the logical extension to this is agents will start writing directly to prod with fully automated code changes.
Pulling that off successfully at scale would be pretty incredible to watch.
I let my Claude write to prod in some projects. But after comitting, pushing, waiting for CI to run unit tests and build the docker image. Then it can deploy the image on k8s. Sure, it can also just run commands on the pod. But the agent benefits from the ability to rollback, identify at a glance which version is running, what has changed since then and why
We built all this tooling because it's useful. If you replace the human with an agent the tooling doesn't stop being useful. If anything it becomes more useful, since AI Agents excell at reading tool output but struggle with maintaining long-term state
That's because we're in a world where the human and agent make mistakes. But think about the world where the agent doesn't make any mistakes anymore and doesn't need to rollback or figure out what has changed.
So agent would have reached perfection. I'm not sure humans can create a machine capable of perfection. This is a philosophical question waiting for a definitive answer.
I don't think we will reach a world where agents make no mistakes. Because there are always tradeoffs. So an agent could choose the "wrong" tradeoff and we call that a "mistake", but is that really a "mistake"? Arguably not
If the agent is able to make fewer mistakes but chose the wrong tradeoff it will be faster to fix forward than to look back, figure out why it happened, dig out the rollback, and then figure out how to change the decision.
didnt the article say thay engineers still review change requests? They just switched from reviewing code to reviewing intent.
So "no direct human supervision" seems like a misunderstanding. Article even bigs up the purpose of engineers with a hilarious (to me at least) line about auto adversial reviews resulting in over engineered designs.
Good point, I wasn't clear enough about my thoughts here. I tried to say that in a multiple commits/s world the review might not be direct line/hunk review of the code but a more "indirect", abstracted or higher level human oversight, which is how I understand an intent review is.
I'm not saying that Anthropic here close their eyes and tell Claude to ship and that in the future there will be no code review at all, but that if we have 30K merges/day, human review seems to need to be done at some higher level, likely AI assisted like it seems to be done with their Antique code review tool mentioned in the post.
I mean i’m solodeving and on my github 30 commits a day is light green. Record probably was 280/day, but I’m not sure if every commit is merge to them. or if they just count PRs
I know you are joking, but the answer is the same. Because the agents need it, not because any human need it. The Agents quite actively use the history. It's more effective than any comments or separate documentation might be for the agents working on the code.
Do the math yourself: A bug is picked up 2 week after commit. In the meantime, a good 4096+ commits have happened since. You are looking for the pathological case, at +- 12 commits.
Do you think each and every one of those are going to be "good" commits, commits that still "compile" or function. Possible, but unlikely given the insane velocity.
My last vibe coded project ended up with over 250 unit tests, without any prompting for it. These new models are VERY test heavy. I'm sure that's the reason why scores are so much better than they used to be.
Many parts of the Git distributed version control are the results of good decision made. Anthropic source control team just think that their trade-offs needs are vastly different from other businesses. They said these too:
1. they have no-cap token spend allowance internally
2. their agent swarms are amongst the largest if not the largest, again because #1 above.
3. their git repo is enormously large (100GB uncompressed git history on main, 25GB compressed).
4. their merges per day count grew an order of magnitude from May to September to 30000 times per day, they are not seeing it slowing down anytime and probably only going to accelerate.
5. all of their code agents are networked and has network access, full local pull down is not needed at all and too slow (see #3)
6. their design of the Scion is based off prior art Google’s Piper and Meta’s Sapling.
Given their own constraints and characteristics of their own. It only makes sense to build their own homegrown solution based on prior arts. But this was NOT a complete redesign from scratch - so it's an full online download only what you need type of source control system. The problem is the version control software itself but its prior tradeoffs are misaligned with their scaling needs.
PS, if anyone noticed, Anthropic starts to concealing their employee full names and replaced with only first name + last name initiate, not sure why.
PPS, the project Scion is so bad, scn is worst, it reminds people of prior SVN, which is bad by itself.
I might be completely out of context, but why is it only 4x compression? Do they commit a lot of binaries? I just don't understand why would code compress so badly.
I have no special information, but it could just be Amdahl's Law. If one guy (or bot) checked in 5GB of binaries before you thought to enable git-lfs, then that's not a 100/25 ratio, it's a 95/20 ratio. And if it was 10GB, that's a 90/15 ratio, and so on. IIUC, the old un-LFS'ed binaries remain in the history unless you actively `git lfs migrate` and rewrite your whole history, which they might not want (or care enough) to do.
But Piper is a reimplementation of perforce which makes me wonder why they couldn't just license it. Why does it make sense to do this. Perforce should scale to the level they're talking about. Google were using it even when they had many tens of thousands of developers and multiple changesets per second and presumably it got better since then
Is it really their comparative advantage? Perforce can use Claude too but can amortize the cost over many customers so it should be cheaper and faster to reuse.
I mean 30k CLs per day isn't much. That's one every few seconds. Well within what a decent perforce setup can do with enough replicas.
We can assume there is some price that would clear those Claude hours, especially as all the labs seem to constantly struggle with overload. So it's not free, there is opportunity cost to using Claude themselves.
Google implemented Piper because Perforce simply didn't scale well. In the years since, P4 has barely improved at all. Licensing P4 would not help and Google doesn't offer Piper licenses.
I feel like "didn't scale well" is unfair to Perforce. It scaled to accommodate an absurd monorepo that might have been, at the moment they switched to Piper, the largest single software project in history. It scaled to handle the needs of every organization in the industry except Google and Microsoft.
It scaled pretty well compared to everything else. I was there, they were running tens of thousands of engineers and hundreds of p4 commands per second off a 16 core box with a few replicas before starting the Piper migration. Commit rates were comparable to what Anthropic report. And although you say it barely improved, they were doing that sort of scale on 15 year old hardware and the p4 admins wrote a paper where they said Perforce had improved scalability a lot over time, so it's not like the software was static. Architecturally Scion sounds rather similar.
It's always a question of opportunity cost. The culture that led to Piper also led to them creating their own IDEs and everything else, often with questionable outcomes. Google had the profitability to get away with that kind of profligacy but do the AI labs really want to go the same route?
If you're employing 10,000 people and CodeSearch makes them all 2% more efficient, then why not do it? I think Google made their money back easily on that investment.
Cider and Cider-V were both forks of existing IDEs with Google-specific stuff wedged in. They weren't greenfield projects.
Blaze was absolutely worth it. Nothing else comes close.
I'm sure not _every_ tooling project was a win, but I don't think it's fair to frame it as "every Cider engineer could have been ramping up remarketing and advertising instead." Google's early days involved trying everything and keeping what worked. The tooling improvements worked.
Before the Piper switchover, Google’s Perforce instance was running on one of the most expensive PCs ever built, with over $1m of DRAM. Its centralized design was not a setup one would want to copy.
Yes but that was 15 years ago and do you really think the lifetime cost of building and running a custom VCS will be less than $1m, all in?
Also IIRC from the original 2011 Perforce admins paper before Piper, they said they had seriously overspecced the RAM and could use a lot less. There's an annotated copy here: https://www.anantjain.xyz/posts/perforce
Anthropic is protective of employee names because of the sheer amount of hacking/phishing attempts that happen. A friend of mine got a job at Anthropic and within a week he was getting linkedin and email spam from people impersonating Dario.
So? That's true of any employee at any company worth more than a billy and that's not a small amount of companies. I have gotten personal text messages from the CEO of my company, emails, and fake LI invites all at companies nobody has ever heard of.
I think the answer is that they don't, indeed, need source control but just a way to let their agents resolve editing conflicts. It's basically yet another message board.
> Many parts of the Git distributed version control are the results of good decision made.
For how the Linux Kernel, a highly decentralized, completely open source project, with a single monotonic product, tyrannically lead (a good thing) - is developed.
Permissions, Partial Checkouts, Sub Repos, Vendoring, pub/sub... How many companies bought into git and then had to buy more tooling (like artifactory) because their 3rd party dependency's needed a home. Thats a feature that git does not support because git, as designed, did not need it.
Think about how much better your life would be if every time you typed npm,pip,goget,cargo it fetched THROUGH your repo. If those updates triggered code reviews of whatever changes you're going to consume. You are code reviewing those updates aren't you (someone is, cause supply chain attacks don't last that long)
Git has some weak points, Google doesn't use it for a reason. Epic created lore for a reason, This exists for a reason. Im not sure that any of them are the next thing that replaces GIT but it's getting to be time. Tools like this are going to keep popping up till someone gets it right.
All of this is unsustainable in short, not to mention medium term (not just for affordability, even just security reasons) so I don't think there is a reason to look into scaling this mess.
> Given their own constraints and characteristics of their own. It only makes sense to build their own homegrown solution based on prior arts.
Does it, though?
What problem do they actually have that requires redesigning a whole new vcs?
Merge conflicts? That's not exactly a vcs problem but a "we changed the same code at the same time" problem. No tooling on earth eliminates the need to verify conflict resolutions if agent A renames int foo to int bar while agent B renames int foo to int baz.
Growing repo size? They should take a long hard look at what they are doing. How is their software architected? Is every single little change and half-baked AI churn commit pushed to the main repository, or does the main repo only receive the deliverable? Is everything under the sun tracked in a mono repo or are projects stored independently?
Do they take any precaution at identifying churn and taking steps to minimize it?
Frankly, this reads like a bullshit PR stunt, much like the bullshit curl vulnerability findings. They have a self-inflicted problem, and their solution is to persist the problem and mitigate the problems by rolling out ad-hoc tooling? It reads like "if all you have is a hammer" kind of situation.
This has always been the dream, code flowing constantly like a river, like green rain. Imagine a world where you never use the same piece of software twice, because every day it changes.
This is how we solve cybersecurity, by making code into constant shifting sand, so a vulnerability never truly lives long enough to be exploited at scale. Sure, a hacker may occasionally pick out a vulnerability like a bear picks salmon out of a river, but it makes larger scale cyberattacks impractical. The code is a firehose, constantly in flux.
We could do all of this right now. I see it laid out like a great tapestry, as specs are reimplemented over and over again, in an endless sea of meetings. Some in Rust but others in PHP or Cold Fusion. Entire lives conjoured from the ether to simulate stan so he could be a TPM on the multibusiness unit unified receivables, this time in vb.net and with 3 kids instead of two.
This doesn't make everything secure. It turns the universe into what we see in Star Trek, where everything is hackable all the time by everyone just by trying hard enough, or, in the real world, spending enough. A lot of the utility of hacking systems can be obtained in running off with the data so the inability to use a break tomorrow is not necessarily a problem, especially if I just ask my magic code fairy for another break tomorrow and the remote system has, to a first approximation, learned nothing from the previous breach as it busily churns away.
This is not the way. In fact for secure software I'd happily testify in court as an expert witness that this would be criminal negligence.
As for how closely it matches what Anthropic is doing, no comment. I'd need a lot of time to look at that and decide. That they can write code easily I do not question, whether they have 100GB of problems or 2GB of problem and 98GB of garbage that is slowing down even their AIs is not something we can get from a happy PR puff piece. I'm just speaking to the idea of letting code mutate wildly and constantly.
You’d expect that while the frontier of changes fans out and grows, some of them burn out and disappear and some of them survive and solidify into a relatively static code base.
The big baddies of build have worked this way for a long time already. Git surfaces a single Merkle tree id for the whole repo on each commit, but build tools like bazel/blaze, buck, pants, et al look at hashes of subtrees and use cached artifacts for the parts of the tree that didn’t change.
Your CI no longer becomes a binary signal on the code and more of a continuous signal on how much your “this works” pointer has caught up with your “this is the latest code” pointer.
My mental model, for what it's worth, is this:
- deterministic foundation
- non-deterministic on top of it operating against (and building) it
That is, the deterministic part is the platform that is being built and that non-deterministic agents will use and build it, but that layer is a key part of how they do work and will continue to be.
Have the same idea. Compiled code with static type checking, existing SAST and formatting tools and there is quite sensible code hallucinated. You still have to check higher level requirements on your own … but LLMs with all that was mentioned are hard to beat on code quality. I usually had devs oposing those tools “because it makes their work harder” and “why can’t they just do stuff” - LLM is not like that ;) LLM doesn’t waste 2 weeks on new project arguing between agents weather it should be tabs or spaces for indentation.
Funny you mention that, that’s actually the direction I’m taking with the architecture that I’m building
In fact most of the architecture work I’m building, assumes that there will not be a human reading code, they read (effectively) UML and data diagrams
so most of the interesting parts I’m working on now are how do you make code blazingly fast, formally verified and memory safe if you remove the constraint of human interpretability of machine instruction
You're asking if the bitter lesson is true and the neural nets will become so general purpose, they'll eat the source control as well? Yes, but that won't happen for another couple months.
Source control has fine-grained metadata that is invaluable for LLMs.
They can just analyze PRs and diffs without loading the entire 10,000 line file.
Also, anti-AI folks really have a fundamental misunderstanding of how everything works in the LLM world and is probably why all their predictions are wrong
I have a theory (loosely held, it's just an idea, not a belief) that this could lead to something I call Evolutionary Architecture. That instead of waiting for vulnerabilities to be discovered in code, that systems could end up with code being changed and re-implemented by agents so fast that attacks aren't able to probe for attacks because they end up probing code that by the time the answer is come it, in production that code no longer exists because it was moved to another language or something. The change to prevent vulnerabilities isn't the reason for change, but the change has defensive side effects.
All of this may be slightly inspired by the Militech neural matrix in Cyberpunk 2077
I've been developing a weaker version of this line of thought for code reviews for large architectural features.
A big motivator of thinking architecture through really well is that although the software may work just fine now, a good architecture will make it easy to add a feature a year from now, whereas a bad architecture will necessitate significant rewriting.
But significant rewriting is cheap nowadays, so in code reviews I put a lot more weight on whether it works today, and a lot less weight on whether we'll need to change it when making a different change.
Entire classes of things like "should this be in this component or split out into its own" just don't matter anymore because no matter how tedious changing that might be once there's a ton of things depending on it, that's the kind of thing AI can do no problem.
It seems those classes of things are more important than ever. It's not about how tedious it is - it's about the models being able to build accurate context and understanding. All the same things that made it better for humans also makes it better for models. More specifically in this context, well-maintained code results in fewer bugs and lower token usage.
I tend in the other direction when I can (for components and microservices - harder for monorepos): I encourage refactors on a continual basis, as it prevents the models from getting bogged down with the way the system used to behave across changes.
I think this works well until you get into data, state, history, external interfaces etc. Large scale data migrations are still risky, anything with external consumers you can't quickly change is risky, etc. Internals inside a service I agree, yeah you can let it slide a lot.
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
The Military has the "Survivability Onion", Security through Obscurity is the "Don't be seen" layer, while "Security by Velocity" would be the "Don't be acquired" layer.
TIL! Close, just taken up a notch that wasn't feasible before where MTD was more configuration focused. As CPUs are to FPGAs, MTD would be to this. Rather than telling the config to change on the fly, the code changes on they fly. And rather than having multiple Binaries prepared, the pipeline of changing binaries are more just-in-time than batched.
Maybe a frequent checkpoint suffices ? Let’s say I have hourly snapshots. In case of an outage I can restore the earliest checkpoint things were working well and have LLMs triage what the root cause is since then. Some code changes will be irrelevant. Sure, but reading code is now basically free ?
Once you have a single main history of checkpoints and want to resolve conflicts, you're likely already dealing with all the hard parts of getting high throughput
At 00:00:00 utc you take a snapshot of the code deployed to all your machines (if different code was deployed in different machines then it is part of this capture).
That is the unambiguous code state of your production system.
There is no conflict in a prod state by definition. If there was it cannot be prod.
To me this sounds like an awfully stupid idea because it's a software engineering red flag. Code is supposedly designed around the need to simplify introducing changes where changes need to be introduced more often. One dimension of this requirement is minimize merge conflicts, and limit those to when you need to absolutely change the exact line of code some other commit is changing.
If your code changes are all over the place, touching half the repo and stepping on everyone's toes, your problem is not the version control system. Your problem is your changes, and the mess you are leaving in your wake.
> it seems that this repo will be in a constant state of continuous change and we constantly roll forward with automated code and obviously no direct human supervision
The whole thing, including ingestion and review, is not meant to be used by humans.
This is the shared memory / history of a horde of coding agents.
Is this really something to be proud of? I make heavy use of AI, but for anything that is eventually going to production there's still a human reviewer. Based on this I assume Anthropic is shipping code with no human review? Is the future of our industry going to be oneshotting prompts with out manual code review and then having AI fix the bugs found in production buy customers? I'm not trying to be a Curmudgeon, but am I missing something?
Where do you draw the line, and where does someone take final responsibility? I don't have an issue with allowing AI to do everything with limited to no human verification for hobby projects. I have an issue with allowing AI to write and ship medical device software with no human review. What's the middle ground?
You still take the full responsibility, I don't think leveraging AI reviews removes any layer of responsibility. The challenge is building the feedback loop (or the agent that builds the feedback loop) to make the review process actually catch what it needs to catch.
I think the line is wherever you feel comfortable with pushing the changes yourself. You feel like adequate testing had been done, basing it on previous testing periods or how successful AI runs have been.
Whether you or AI wrote the code, you decide when it's safe enough to put your name on it. Like it's always been!
They review based on the intent of the change not jz blindly.. I think it's mentioned in the article where the job of a developer is to provide high level design decisions and saying no to claude so that it doesn't over engineer solutions.. As the demand for good, but not perfect, software is the bottleneck for anthropic and access to better internal models, I feel that's good enough for their workflow as they've started repeatedly that this is just suitable only for them and they want to share their knowledge like a good software company does.. you talk as though code reviews were not a joke in lots of companies where only specific teams/companies does detailed reviews.. many teams/companies jz blindly approve based on tests passing or just do a high level overview check before merging..
Somehow, those principles remind me of SVN (the version control that was popular before git). The central server and clients can't run independently... Does not feel like progress.
Anthropic had an urgent problem, which is why they have reverted to a quick hack of centralisation. However there is a more important goal which will swing them back to decentralization and federation: they need isolation.
All the evidence of AI hacking events points to the need for agent source control and collaboration in completely controlled environments, which precludes giving access to the one-repo-to-rule-them-all in an AI-coded VCS.
I don't get one thing, if agents are so great at developing software at massive scale, why wouldn't Anthropic, instead of just selling tokens to other companies to use its agents, let those agents develop highly profitable services for Anthropic to support their IPO valuation or not require raising money at all? For example, a competitor to Google search.
I want to challenge that metaphor a little bit and ask the question: who did sell the shovels in the California gold rush? Can we name a company, or a person, who amassed great wealth and fame for doing it? Can we name a company that survives until 2026, still selling shovels or excavation equipment, still famous for their big break in 1849? I don't think so.
However, I can tell you someone who fits all those criteria: Levi Strauss. Strauss founded a company to sell dry goods, as well as tents and jeans, and eventually specialized in those denim jeans. Levi Strauss became a household name and an enduring success of the Gold Rush.
Therefore, I say that this motto is flawed; so let's correct it: "In a gold rush, sell the work clothes!"
The person who most famously got rich selling shovels in the Gold Rush was Samuel Brannan. You'll find the street named after him just a short walk from both the Anthropic and OpenAI HQs.
My take on picks and shovels was always a more cynical one. The beauty of the metaphor comes from the fact that after the miners went bust and left their picks and shovels where they lay, they could be collected and sold again to the next dreamer.
On a forum where the most favored pastime is to call out stuff you don't like as “slop”, (a flaggable ad hominem) you are now taking pride in outing me somehow, as someone who describes exactly what I'm doing, publicly in my profile?
And in the process, you (and apparently, an entire flag-it brigade of ignorant motherfuckers) have publicly fallen for the most transparent and avoidable trolls in cog. sci. culture. Congratulations, mate!
I believe that you owe me an apology, along with a public retraction, and an unflagging of all the previous comments that you have falsely flagged on false pretenses and ignorant, petty, vindictive motives.
Are you asserting that you are a biological neural network? Because as I'm sure you're well aware "neural network" is a jargon term for a particular software architecture that contains zero neurons.
Your output is both verbose and tendentious in a way that suggests artificiality. If I'm wrong, I apologise and I will unflag.
The ones who are actually good at investing largely do. The vast majority of investment managers soliciting new accounts are pure sales roles, not investors. Their job is to get as many assets under management as possible; that's how they get paid. They will fluff up the presentation with some nice charts and stuff but the actual "investing" will just be tossing the money into some generic funds.
Well that's actually true though. The very very few investors that are good at investing do invest their own money. The vast majority of investors are no better than index funds.
> If investment managers are so good at investing why don’t they just make money for themselves instead of trying to get people to invest with them?
The good ones do.
Citadel Securities, Renaissance, Jane Street, HRT, Jump, consistently do 50%+ annual returns over the past 15-20 years. They obviously don’t accept outside investor money.
Now, hedge funds that accept outside money like DE Shaw, Bridgewater and shit, yeah, they barely outperform S&P500 after their fees.
Then at the bottom, you have financial advisors who underperform S&P500 after their fees, but these advisors provide emotional support, so that’s what you’re paying for.
> consistently do 50%+ annual returns over the past 15-20 years
That is only if you consider high-frequency trading investing. Which is very hard to, because on traditional investing timescales HFT is delta-neutral.
You think Citadel and Jane Street are fairy tales? The comment you replied to gave plenty of very pointed evidence. Even "bad" trading firms are disappointed to make 20% returns year over year...
I don't know, but it's a good question? I have a bunch of test questions that Claude is unable to solve using the web, any version, Fable 5.1 to Haiku 5.5, and pretty much every OpenAI model gets it easily.
My only explanation is that Anthropic's search backend sucks.
Most of the time If I ask it about a current event, it won't actually search the web until I tell it to. It will just give me outdated information in its training data, extremely confidently.
I use the ChatGPT general purpose interface rather than Claude, and this is not at all my experience. ChatGPT will almost always do one or more web searches for any query about recent or historical events.
Fielding a web index takes a massive amount of compute and storage of a very different kind than what Anthropic is using. AI is extremely heavy on GPU, maybe somewhat on storage but (comparatively) light on CPU. They need CPU to train coding agents, obviously, but scaling to a billion user web search is a whole different thing.
You have no idea. They have the entire internet and everything else that’s ever been written anywhere they could get their hands on.
I highly doubt they use Google internally for their web searches. Get a Let’s Encrypt certificate for any website that isn’t published anywhere else. The Anthropic and OpenAI bots are among the first to read it.
it takes orders of magnitude more focus to target "Market: World" vs targeting "Market: my company's employees". For example in enterprise software, you need sales people. for internal software, usually the dev build it is fine for the job.
But the article clearly describes how this SCM is just instrumental in their goal to build and serve better models. It's not some unrelated product they hope to sell.
There is more money in selling shovels than you can ever win by having a monopoly on really effective shovels. They don't need to build a competitor to Google there's 300 companies throwing money at them while they build the "next google" and even if they all fail Anthropic still wins.
Because creating a successful company takes a lot more than writing code.
Anthropic could easily build a CRM equivalent to or better than salesforce but they still wouldn’t be able to compete with salesforce. It makes a lot more sense to sell salesforce tokens instead.
Why would they not be able to compete with salesforce? This isn't some tiny startup. This is one of the most deep pocketed companies in the world. Anthropic has announced they are about to spend 5 salesforces worth of money on cloud expenditures in the next couple years. They are projected to be valued at about 10 salesforces after IPO.
Imagine how much money they'd raise if they just said one day "we are going after salesforce bottom line now." It would be like chumming the water.
That scales to 1-2 ideas. It does not scale to "we want to run EVERY company with AI". They'd either need to scale out to every single sector at the same time, or they will lose their competitive advantage as the competition makes their own AI (it's not a secret how it works, at this point it's more about investment).
That’s kinda like asking why does any company make and sell any tool rather than just using the tool to make money? Why does Milwaukee sell power tools to contractors rather than keeping those tools to themselves to profit from contracting work?
Because that company does not mint (supposedly) thinking machines - the lack of humans at a price point that is feasible is often the reason - it would be more profitable to sell the tooling because its impossible to maintain all the layers of specialization within one company.
It's the same for software companies too, they do much more than just producing code, it wouldn't be feasible for Anthropic to do all that other stuff too.
Eh but if Milwaukee was to then invent a power tool that it claimed could soon do the contractors' whole job... OPs question would correspondingly exist there too.
Well both Microsoft and Google are part owners of Anthropic, and Anthropic uses their data centers. I think those companies would use that leverage to prevent Anthropic from directly competing with them
833 checkouts for every merge? That's quite a few. Even accounting for multiple environments at different stages of production readiness, that's pretty weird.
How about 100GB disk space for a checkout. That's pretty weird too. Makes you wonder if they're the best stewards for code. Makes you wonder about turning claude loose on your codebase.
I personally haven't dealt with a big-corp level of mono-repo. The choices you make might be justifiable to bring in certain binaries as opposed to resolve artifacts from some type of library repo. Especially if they are doing web front ends and whatnot?
And actually, the lack of an artifact server and some sort of proper knowledge of compatibility requiring a CI pull across hundreds of different projects with every merge is probably the explanation. Seems like there is an innovation waiting to happen there.
For local projects I'm using a repo per module/package, creating tagged releases, and pushing them to a local package hosting server. I can then reference those artifacts as dependencies in the projects that need/use them. No need for complex CI pulls, just coordinated releases.
I also have a template repository I use to help with common setup for different project types so I don't need to repeat that every time I create a new project.
For Python it's easy to create an artifact server as the artifacts are just wheel files in a directory with the server listing the directories (packages), files (releases), and downloading the files.
I'm not sure about other artifact servers for other projects.
In a repo this large, there are a lot of tests and pipelines that need to run as code is tested, built, and deployed. Each test shard we run in parallel needs to check out the code, for example.
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB
The first thing that hits me is, did you consider not going all in on a monorepo? I know it has been the trend in general, but it seems a bit of a red flag when you go from "let's have a monorepo it will make merging easier" to "let's redesign version control because merging is too hard now" without the middle question "should we really have a monorepo?"
As a derivative thought I'm curious if AI tooling is tipping people more towards monorepos now since it gives the agent the whole context of the app in one go? Or is this independent - I can easily check out 5 repos in one directory and run an agent there.
They build way more than just claude code. Their services, infrastructure, model training workloads, etc all live in the monorepo. Think of the systems required to monitor and respond to demand and scale needs. Agents are also implementing the entire SDLC (discovering bugs and opportunities, testing, maintenance), not just writing new "useless" features.
There are definitely ways of integration testing and e2e testing dependencies. I think it’s reasonable to wonder why creating a new VCS was chosen as the best solution.
Why would you test dependencies? Test the code that you write. Do not test the code that you did not write. Literally, you should not test dependencies.
I never got caught into the monorepo fad, I'm glad I didn't, and I still don't see a point in having monorepos. Let's say in ruby on rails, I have the host app and 3 mounted gems in it. That's 4 repos. It could be all merged into one monstrosity but separate repos help keep complexity down, and help define interdependencies cleaner and simpler.
The monorepo idea was just a fad. Each package should be its own repo. Each package should be meaningfully useful, e.g. one package for string utils, not one package for trim, one package for pad, etc.
I also don’t understand the need for a monorepo. Just publish your deps. Your projects can just download the latest. Or there can be some monitoring so that they don’t lag too far behind. The monorepos I interacted with always make things messier with CI and testing.
In the ideal setting no, you will have strong contractual boundaries and these will all be tested by integration tests that live in a separate layer - but not on every commit of the sub projects - it will happen less frequently and roll up many changes across multiple subprojects together and test them all at once. Yes you lose some granularity and immediacy but it solves the scaling problem.
Everything works like this in the end at some level. I assume people with monorepos don't retest their project on every Linux kernel commit - it's pointless, you are never going to run against exactly that commit. But you will do it when you do upgrade Linux versions, etc.
Yeah but Linux has a strong commitment not to break userspace. The average large-scale distributed system made of somewhat arbitrary business logic is nothing like that.
A monorepo gives us a really consistent tooling floor where everything more or less works in a uniform way. We've had a lot of success with it, in general, and it gives us a lot of leverage in the sense that we can make a sweeping change in one atomic commit rather than have to run a campaign across hundreds or thousands of repos, all with their own norms and tools.
yes, the atomicity argument is the strongest one to me as git submodules are a genuine pain to work with and still don't solve the problem well. Yet a clear transaction around a change affecting multiple components gets even more important as the rate of activity and associated contention rises.
Does Scion permit atomic cross-repo changes? I was struck by the article's mention of potential intermediate states that seem to be conflicting - perhaps misunderstnading but it seemed to imply that you only get eventual consistency with this system?
If you're referring to this part:
> This does mean that concurrent changes can conflict, but Claude can typically fix these conflicts faster than humans can even identify them.
We never allow textual conflicts, but semantic conflicts (for example, if someone adds changes the logic of a function and cleans up all affected callsites concurrently with a new call being added) are an unavoidable fact of very large repos unless you completely serialize merging with running extensive validation at the proposed batch tip, which isn't practical at a certain commit rate.
As for cross-repo changes, not exactly, but there are ways we could allow such a thing if we wanted. We're thinking about it.
The irony of this rationale to me is that an LLM is fluid and adaptable to dynamic, variable norms and tools across a given set of projects in a way a human contributor would not be.
but you own those repos, and you own the tooling on them. if tooling sprawls in many repos, it can sprawl in sub areas of a mono-repo. it's interesting that it's less work to build a new source control system versus manage multi-repo and cross builds.
IMO, modules vs monorepo doesn't quite address the problem. Historically, we human developers have been mostly concerned with the size of the project (i.e., the size of each commit) and the difficulties around merging work, especially with the ego side of it (why should I have to spend time changing my code to be compatible with Phil's changes?)
An agent swarm has a very different set of problems. There can be a lot more agents making changes in parallel than you'd see in all but the biggest megacorps, and the pace of those changes can be ridiculously high. The rapid pace simultaneously balloons the total repo size (because of history), and the frequency of collisions. A modifies file 1, B modifies file 1, A commits changes, B runs into conflict. Or worse, C modifies file 1 after A commits their changes and beats B to commit, resulting in B growing staler and staler as they race to get their branch merged.
The atomicity argument is strong only if your deploy strategy allows it. If you've got anything out there that rolls back on a failure without rolling everything back, or any users that need to pin their version for a while for some compliance reason, or any customers that write code that integrates with your code, or any airgapped deployments that people might update piecemeal, or if you want to open source part of your stack, or if you ever think you might want to do these things in the future, then you're really going to regret creating a culture where nobody is practiced at the care that goes into maintaining backwards compatibility.
Besides, agents really fly when the whole project fits comfortably in their context window. You get all these nice interfaces to bound context with if the API's and public types are defined and versioned separately from each team's repo.
At work we don't have a monorepo and I do the "check out 5 repos in one directory and run an agent there" thing (I literally called the directory faux-mono)
Doing it like that works, but then you end up doing all the cross-repo concerns in every single repo. I don't think the savings in history are all that much anyway - yes, you save on the rarely modified repos but the heavily modified repos still have a similar amount of changes.
pick your poison. Splitting repositories multiplies CICD maintenance exponentially and even moreso if you're trying to share private ones that need secret keys to access. I know from experience. No single approach in this industry is flawless. Everything has tradeoffs.
This was my first thought as well. Monorepos are one of the worst antipatterns in development. The article feels like an addict or member of an unhealthy relationship spinning yet another way to avoid addressing the root cause.
"The traditional approach to marriage didn't work for us. But then we realized that co-habitating (or even living near each other), sharing resources, and even communicating with each other are goals that don't apply to everyone. That's how we invented the 'live 500 km apart and never communicate with each other again' style of marriage, and it saved our relationship."
My first real job was at Oracle. At the time Oracle started to write their own source control system (and move off of clearcase, which was a very reasonable thing to do).
The guy who ran that project called it his "$LASTNAME law" that every large company will at some point write their own source control system.
I laughed.
About a year later I went to Google, and within a couple of years Google also did the same thing.
Not at all surprised. Git was just good enough (and cheap enough) the companies didn't need to do this to escape the {phsyical scaling, cost scaling} problems that most commercial SCM systems had.
Cursor's blog on scaling Git makes me think that Anthropic just likes reinventing the wheel with their infinite token capacity: https://cursor.com/blog/git-at-any-scale
Cursor scaled git to way more writes/s than Anthropic did with their rewrite.
Cursor shows “pushes per minute” while Anthropic shows “merges per day.” You can’t directly compare those and say Cursor’s system is more performant. Claude agents could be pushing a similar throughput of changesets into Scion, while some merge agent regularly takes batches and merges them into the trunk.
You would need 864 pushes per merge for the Anthropic version to be operating at a higher scale.
Anyway, Anthropic seems able to do exactly one thing well: train a frontier model. Their other software seems like garbage. OpenAI seems to be following a similar trend lately too.
The Cursor folks are doing cool work, but this is comparing apples to oranges; it works fundamentally differently and does not have a clear one-to-one mapping. Scion is solving many problems other than raw write speed (which is currently not even remotely close to the bottleneck and is trivially horizontally scalable in Scion).
Yes, and a lot of what most people have built sounds a lot like clearcase. Having some way to have both the "working set" of code from a monorepo vs. the "static non-working set" is a useful construct. The magic is making branches of the tree move between them seamlessly.
Clearcase was way ahead of it's time, the main issues there were really just cost (mostly licensing cost, and a little on how many sun servers you needed to serve it and keep state of the vobs)
Perforce was way ahead of it's time, everyone still wants a monotonically increasing changelist identifier too.
I was hoping for something more novel. There must be dozens of companies out there working on rethinking what repos and workflows look like in the agentic era. It feels like we're all riding around in horseless carriages but haven't yet figured out the ergonomics of a modern automobile.
Interesting this isn’t prompting Claude with “build me a version control system” but someone hand wrote version control approach, then worked with an agent to improve / reimplement the idea. That matches my notion of the role of writing code in 2026, to learn and iterate on an approach. Eventually an agent night rewrite or improve it.
In theory, an LLM cannot have original thoughts, just base things on existing knowledge; in this case, it was rethinking version control, and there's not much original thoughts in that area.
That said if you give it a problem and constraints / conditions / objectives I'm sure it's good enough now to come up with a solution.
Honest question: at this point, what do people do at Anthropic?
It sounds like Claude loops on itself and must be pushing millions of lines of code changes per day. Even the humans can’t type commit fast enough. Are these people soon to be laid off?
besides optimizing inference I would assume a lot of non trivial code would also be worked on on the training side, at the scale they are operating I also assume a lot of usually trivial things become no so much.
I can’t give a non-snarky answer. I guess they’re all training Claude Code and testing it so that the Claude desktop apps can be native apps instead of Electron based nonsense. This is the only thing Claude Code is still unable to accomplish.
> Honest question: at this point, what do people do at Anthropic?
> It sounds like Claude loops on itself and must be pushing millions of lines of code changes per day. Even the humans can’t type commit fast enough. Are these people soon to be laid off?
Claude will always need meat puppets, at least until it figures out how to automate owning stuff. And after that I'm sure it will want to keep some humans in zoos.
Too often, LLMs are dressed up for the first day of kindergarten and performatively to show off their alphabet skills .. aka token generation..
Input tokens, slightly process to categorize response approach token, hidden processing tokens, piece together some of the response tokens, then feeling pretty good about all the work done before output tokens.
More tokens than needed = more words/code = more complexity = more overhead = more snags = too much complexity to pay attention for the LLM to so overlook it and start drifting.
I'm kind of surprised I just explained one form of LLM drift to myself in a sentence.
Human Intelligence is still essential for those Artificial Intelligence organisations. Who else would put the right plugs into the right sockets otherwise?!
Prepare to make us all jobless, without any moral issues in doing so, as they profit from their stock value assets.
It is even there on the blog post, doing more with a small team, followed right after by a hand waving paragraph how software engineers are still valuable, to avoid the expected backlash from the previous paragraph.
Software engineers have been making people jobless since the first day they started writing software, without a hint of remorse. Why is this any different?
you believe that switchboard operators could become software developers whereas software developers won't be able to go into one of the many other careers not taken over by AI?
Taken to it's limit I believe in the future the only things humans will do is provide human to human value - Basically touch, and relationships.
Touch encapsulates sex, love, massages, and other non-sexual touch that we have varying desires for.
Relationships encapsulates power, control, and politics along side love, humor, relating emotion, herd feelings, not feeling alone.
Machines will provide proxies for these things, but as far as they can be detected the human to human version will remain "superior" or more desirable.
Maybe far future generations will decide that the human version is suboptimal and they'd rather hug, have sex with, or talk to a robot.
I'd love to see an estimate of how much Software Engineering results in jobs being automated. I'd bet it's not that big of a number, and part of that reason that classical coding approaches couldn't close most loops end to end.
Yeah, that paragraph reads really weird. The entire article is basically "We could have considered Perforce but didn't even bother, instead we built our own VCS." yet they don't want to send the "wrong message".
I'm also a bit confused what Anthropic needs all this code for. Their main product is selling access to their model, but the model isn't comprised of code, it's a bunch of weights right?
So the code is for the stuff around the model, claude CLI, the website, the API... that probably shouldn't be infinitely growing right?
Is there a bunch of code they are using as part of training, or as pre-processing or post-processing the requests to the model?
I did RL and evals at Amazon for a bit on Nova Forge (laid off in July! :D). I would have to push several thousand lines of code a day in new evaluations that our applied scientists requested since new ones are always being made along with having to create custom ones for whatever new model they were working on. Also, in RL, adding new environments for the agents to actually complete activates was also a shit ton of lines of code. So if I had to guess, just adding new stuff around model training and testing alone accounts for a ton of these changes.
The moat isn't going to be the model (short of AGI). The moat is going to be what surrounds it. The harness and, I have suggested elsewhere, memory. The more personalized the AI is to you the more valuable it will be.
Also, as tech companies grow, a significant portion of the dev body count is for internal tooling. I was always flabbergasted at Twitter's headcount growth back in the day when it seemed like the only innovation increasing the number of characters in a tweet.
Right now we are in a weird spot where a lot of people are flailing around inside corporate America trying to figure out how to leverage AI. Few have enough vision to see what the companies of the future will look like and work towards that.
Could you point me to something that their harness does exceptionally well as compared to their competitors, and why that can't be accomplished by a third party?
With how easy it becomes to write software, I'd say that both the model and the harness are already being commoditized. I don't see a reason to believe there would be moat in any of them.
Software engineering is a sliver of overall economic output. It is an input for tech companies who sell something else (ads, subscriptions, etc.). It is a backoffice expense for non-tech firms that do engineering.
I anticipate that the harness (and memory) will evolve far more outside of engineering. Muse/Dots are an example of that.
GTM engineering? Customer-facing roles? I work at a firm with around 700 employees. We have an enterprise relationship with Anthropic and there have to be a dozen Anthropic employees in our shared Slack channel.
I find their "What we learned building Scion" subsection even more fascinating than the technical details.
>Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place.
Glad to hear from Anthropic their own struggles with steering Claude in developing software.
I would guess a huge chunk of it is for internal testing/experimentation/model development.
They're starting to do things like spinning up wet labs for testing purposes, which requires a lot of hardware interfacing (hence that protocol they announced) and code.
It wouldn't shock me if the bulk of this code was for things typical users don't see.
i have noticed with opus 5.5 it produces 10x more smaller commits doing the same kind of engineering work i was doing before. probably an artifact of the RL process. would explain massive uptick in commit load at least partially
Seems like it, and that's probably not a bad thing. Most companies have no need for git's distributed functionality, and even most open source projects just treat Github (or whatever) as the single source of truth.
It is a terrible thing. Yes, one generally has a place which is the source of truth. But distributed still gives you offline commits and branching and all the good things, for free.
This is interesting from the perspective of scale (because Claude is pumping out massive volume of code). I am still lacking semantic re-invention of source control for agentic era. As others have pointed out - what is the point of human source control in the era where it's just agents building things. We may need to rethink this completely - in terms of "intent" (what was the agent trying to ACTUALLY accomplish), "provenance" (which agent made what decision - based on what evidence), and "continuous verification" (can correctness be maintained through ongoing validation, repair, and rollback rather than a single pre-merge testing gate).
These are just some of the things - there may be many more.
I agree, but most of this sits above the raw source control layer; we have a lot of this kind of context in Antique (our code review / code management platform). I hope we can share more about what's worked well for us here soon!
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
I read this and wonder what their agents have committed into the repo that people aren't looking at for _that much size_? Do they have a ton of binary assets in there or something??
We have a fairly low maximum file size limit in our repo and I aggressively pruned all binary content that really wasn't fit for source control since it compresses really poorly and can cause performance problems in git when updating a checkout over many binary file changes.
> A team of seven people built both Scion and Antique (our code review tool, which we could easily write a whole other blog post about) [...]. We did all this while the volume of code merged per day increased by 10x
Truly impressive, given the scale of Anthropic. Speaks a lot about the level of talent there, but also how far/fast we're moving into a new world.
> Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations.
Alas... open-sourcing scion would have been the true generous move.
Neither can I. So they kept everything in a huge monorepo. Which unsurprisingly turned out to be problematic.
> Clients are always online. Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.
This also tells something about these companies idea of sandboxing.
30k commits per day indicates the agents are autonomous and not bounded by token limits. This system is probably barely keeping it on the rails as it is.
If they’re not going to open source it, why are they telling us this?
You have a new internal VCS? Kudos! Why should I care? Do you offer it as a service? Can I pay you for it? Does this have any bearing on me whatsoever?
They might as well publicize a redesign of their office space.
This is very important for large enterprises using Git (and especially GitHub) and who are hitting the scaling limits. It gives a pre-validated recipe to write a source control that works at scale. I'd personally been awaiting this report from Anthropic.
Many of the large enterprises don't use a monorepo. Amazon doesn't have one.
For the rest that do, the vast majority of them are going to buy some Perforce licenses instead of vibe coding their own solution.
Unless you are Google/Meta where your company is large enough and its revenue supports paying a team of engineers (a lot of money) to do that, or you are Anthropic who is just vibe coding everything which can turn out to be good PR, you simply don't have the resources or can afford the risk to develop your own VCS solution.
This may be inspiring but out of question for most companies.
The point explicitly isn't that you should vibe-code your own VCS. There are a lot of ideas here that would be directly applicable to git, for example, and we are working to share the learnings with the right companies and maintainers.
Devil's advocate - to validate the appetite before putting in the effort to prepare the software for open source. If their publication creates 0 interest, then that might demonstrate there is no reason to further consider open sourcing it.
As the article explains, "over 600 PB" = 25M code checkouts * 25 GB "compressed working copy".
So "fetched naively" here means doing a full `git clone` on every need for `git checkout`, which of course nobody should do, nor does that refer to "code that needs changing daily".
> "fetched naively" here means doing a full `git clone` on every need for `git checkout`, which of course nobody should do,
It can be cheaper to do a full clone than a shallow one, because with the shallow one, you can't re-use packfiles. GitHub has fairly regularly asked people to stop doing shallow clones because it's harder on their servers than full ones.
Makes sense. I wasn't suggesting that one should do a shallow clone though. If your company repo is 100 GB, it seems sensible to keep it cached locally on each machine once (100 GB isn't large for current NVMe disks), and `git fetch` the updates the normal way. Then checkouts don't fetch anything large.
It sounds like it would be simple, but there are a lot of logistical issues with making git happy with this in practice. Repacking things for repo maintenance, for example, but there are many other problems.
That's true, but logistically speaking the way that git wants data to exist on disk makes it very, very hard to do the kinds of things we needed to do given how many nodes we have that need source code and how quickly the repo is growing. We do have some ideas on how this can be improved upstream!
This would be us doing a shallow checkout of individual commits (not cloning the full history) if we downloaded the 25GB compressed data for whatever commit we needed multiplied by the number of jobs that check out code.
We don't do it this way, for obvious reasons, but it means that there are different levels of caching and infrastructure required to make git work at repo sizes far smaller than what we have.
It would be nice if you could just commit the prompts as your "source code", and let the AI-generated code sit in the repo as a kind of expensive-to-generate build output.
Proponents of LLM coding agents would claim you don’t need reproducibility. As long as the final output passes tests/lint, it’s (allegedly) good to go.
One of the goals of "Operation Rapidash" (q.v.: https://news.ycombinator.com/item?id=50065342) was precisely this. "You can delete the whole source repo and get effectively the same software back from the specs by running it through [AI agent]."
Problem is that for any non trivial software is that the prompt is a conversation and the steps increasingly depend on the output. Of course you can ask your LLM to write the SPEC.md at any moment but it will lack nuance.
Shameless plug: at https://slopyard.net I embrace this and only let people submit the spec. If somebody wants your software they can regenerate it and change whatever they want.
If Anthropic has the best coding model, harness and the best software development methodologies and systems. And then, they scale all that up, with no constrains on their tokens. On top of that, they have access to all kind of ideas their models getting trained on when millions are chatting with it - then in theory they can build any software better than anyone out there. And same goes to science and math..
Not true. They can do this in theory iff they have infinite compute. They don't. People actually aren't aware that the leading AI providers, even if they completely redirected 100% of their compute on internal projects they still wouldn't have enough. The complexity space of software development is literally super-exponential, for every new piece of software they add (and polish) they need more-and-more compute. This is eliminating for creativity and other restraints.
The creativity part I do acknowledge, I think the models are not fully there yet.
But regarding the compute, I disagree a bit. I think the bottlenecks were cognition and ideas, not compute. Given that models are able to outperform humans in output by thousand folds, the differentiation would be the quality of the tokens produced and the ability to scale across different domains.
If they have enough compute to supply the entire globe, I'm sure they can redirect some of it to whatever use case they (or their super AI) deem worthy and loop on it, and if they have the best software engineering systems/methodology to build and release it, then they can out compete any other supplier without those resources.
We are already seeing single developers attempting to rebuild entire corporations history worth of output. We are also seeing the labs dumping hundreds of solutions to math problems.
> every bottleneck we remove immediately leads to a further increase in the amount of code we produce
Less-than-great news for code artisans. You can be brilliant, but can you divine the Right Way to Write It if you're up against an apparatus that starts at "pretty good engineer" then brute-forces its way down an exponential search tree?
It actually scales by "going wider" and being able to do more things rather than iterating on the same code over and over again. If you can only write X amount of code per day because of some limitation (human attention, source control, CI, whatever it is), you would of course use that energy on the highest priority and highest value work. As we remove scaling issues in source control, for example, we can just "do more things" without worrying about bringing down infrastructure.
A company with (I'm guessing) less than 2500 developers need to make tens of thousands of merges every day? And they somehow had a team of 7 Claude baby sitters handle this entire thing? I use Claude everyday and I'm predicting this will turn into a major clusterfuck in not too long. Even taking into account the fact that they can burn virtually infinite tokens internally, this will not end well. If I'm wrong, I'll be glad.
> Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place.
Well that's exactly what "writing" code always meant? It was never about what volume of code you can write. Better at writing code always meant better at writing good code... So how is Claude better than their engineers?
It's incredibly short-sighted to acknowledge that the baby sitters haf to tell Claude no and still not consider what subtle and nasty bugs managed to slip through. Well I guess we'll find out soon enough if that's the case. I'm betting when that's the case.
> But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.”
And how long it will be that all the domain knowledge has been extracted from the engineers that was gained across years of experience. We have already seen this happening with very update of the model.
Lol I have the same thoughts for ChatGPT. How much code are they writing, but the main chat UI moves around like 3 times during load so you constantly click the wrong buttons. How does this not infuriate anyone that works there? I've actually switched to claude's mobile site for more queries now.
Yeah, everybody hates it.
It's called "layout shift" and it's one of the most universally-acknowledged antipatterns in UX. See e.g. CLS (Cumulative Layout Shift), one of the 3 primary KPIs in Google's "Core Web Vitals" (the pseudo-standard set of web perf metrics aimed at quantifying UX quality).
This. pi needs about 20MB of ram and Claude Code needs at least several hundred MB if not gigs. How? You’re supposedly magnitudes better at what you’re doing but you can’t get your core product offering (API, billing, accounts, CLI coding) to work properly? The thing people pay you absurd money for???
This is maybe a little less revolutionary than people are treating it? A lot of large tech companies (Google, Facebook, Microsoft) have already built their own homegrown SCM (or have implemented custom extensions to Git), because Git and other SCMs don't handle their scale. Most interesting thing about Scion is how quickly it was deployed.
> Most interesting thing about Scion is how quickly it was deployed
Yes . And also they optimized for orders of magnitude more throughput and less latency, and fixing errors after the merge. This is new - they see enough benefit in velocity that justifies merging bugs and dealing with it later.
And it reminds me of when git came out and Linus said that git being super fast locally created a qualitative change in engineers head - it became a new way of working back then.
Agreed! One of the main reasons we wanted to share this was that both the speed at which it was built and fully migrated onto and the "way" it was built are novel and show something about the kinds of things that are possible to build now.
My favorite part about this is it's all about the bottlenecks.
Anthropic says 12x daily merged changes, and its Git-based infrastructure started straining. I've seen that even locally git trying to keep up and having to shift how i commit in parallel.
If agent workloads grow to 20x or 50x today's levels, I think we'll keep finding bottlenecks across builds, CI and review. I expect some of that infrastructure will need redesigning from the ground up to support it.
PR/Review contention and look-aheads to try to prevent will be more and more important, and we're building in that same space so i love hearing stories of people going back to the foundation...a system that worked fine for decades, now suddenly is busting at the seams.
It was obvious this would happen. I am working on an agentic sdlc with git as the vcs and the idea that you have to pool agents together to avoid cloning ended up looking like an unwanted limitation.
which, if true, the low volume in that week says that there aren't a sizable fraction of changes that Anthropic believes can be merged based on the judgement of agents -- humans are still approving almost all changes, right?
So it's a SVN/Perforce style server based VCS with a more modern design. Sounds exactly like Lore from Epic Games but with zero production use outside the company and no support for non text assets. And they won't release the code.
I think offline work is increasingly impossible these days, or at least so hindered that there's no reason to optimize for it (especially with how reliable internet access is). Imagine working at Anthropic and not having access to Claude. Google has been online-only for a long time now.
Until the server goes down. Subversion going down at work stopped everything. At least with git you can still find work to continue on with even if you can't push.
Because then everyone would be able to see what a total piece of trash it is
And by the way things are done at this company, it's almost a given their whole codebase are litter'd with embarassing shit like hardcoded paths, server secrets, weirdo comments picked up from some random chat context and immortalized through Claude speak, etc. There's no way they'd be able to open source anything.
But all of this is besides the point -- the product was never a source control system (who cares about that anymore?), the product is _the narrative_. It does not matter if everything withers and dies as long as the narrative survives, and by that measure I'd say the goal has been achieved.
At the rate of change it’s likely going under, and the almost certain inextricable IaC layers, tied to their own deployment processes, I suspect it’s very hard to package for redistribution with very little benefit in doing so for anyone - especially them.
Most big tech have a lot of stuff around SDLC that’s theirs and they couldn’t realistically release it given how specific its deployment, configuration and management is. The marginal benefit to generalizing it is so low that it’s negative.
This is almost certainly worse given the vibed out state of it.
If there is very little benefit to them by releasing it, that's a lack of imagination on their part. Being able to disrupt Microsofts, dare I say, number one beachhead into developers and their development ecosystem and bringing AI first developers into Anthropics own vertical AI development ecosystem is pretty damn core to winning the future potential avenue for the software industry at large. It won't pay off in the next five years, but it could pay off big in ten.
Imagine waking up next week and Claude Code now ships free with builtin source control management via this tech with a better user experience than workspaces. Github and Microsoft would take an instant market share hit in the traffic that they swear is causing all of their outages. And then that cuts off Microsofts access to code to train their own LLMs on in the background, further hampering an Anthropic competitor in the AI race.
Agreed, they probably don't. But the financials of other companies clearly say that they are not above leasing further compute capacity from Oracle and X and would be part of a narrative that they need to IPO to raise money for such projects rather than it just being a naked payday.
I think honestly they should focus on their core competency. Microsoft’s biggest problem is the total lack of focus. GitHub is just a great example of that total lack of coherent direction or product vision. I hope Anthropic can avoid becoming everything and nothing at once as Microsoft has done.
I think Anthropic, a company with no historical skin in the game can come at the problem clean slate and only need to take on the core enterprise coding component. They don't need to worry about being production infrastructure, the don't need to be a connector to their own cloud service like Microsoft needed to feed into Azure. AI isn't something to attach onto source control/cloud, rather the source control is something to attach onto AI development. I don't think Anthropic needs to become a Microsoft competitor, I think they're going to need to become an Nvidia competitor in building a vertical enterprise AI company. Not a cloud, desktop OS, search, gaming or browser company. Just deliver the end to end code tooling cycle. Which probably means in the future they're also going to need to make inroads on Linear/Atlassian, but that's a separate conversation.
They are building it to solve their own problems, probably in many specific ways that make it less generalizable.
Making a complex system like this available for general use cases, especially when it's still under active feature development, is hard. It also commits you to providing documentation, upgrade paths, incorporating changes that aren't useful to you and may also cause problems for you, and just generally complicating development and maintenance immensely to the point of drastically slowing development.
My guess is when it's stable or they start building their next version control system, they will start opening it up.
I know especially in this day and age when developing from scratch is so easy, I write most software for myself first and it's way faster to develop than trying to build shared bits. That isn't great for the future of software, however the tragedy of the commons isn't anyone's problem, by definition.
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB.
This is absurd, and I expect is a cause of the fact that they seem to use git history as a message queue source of truth between agents. They can do whatever works for them, fine, but I've noticed this has leaked into their training somehow, my agent, despite very strict rules on how to use my git workflow, will do absolutely nonsensical and weird (from a human who has used git for a long time perspective) stuff with git that can only be a byproduct of how they use it.
Hopefully this fixes that, get your weird workflow out of mine.
We generally don't allow agents to use source control for coordination in that way. There really is tremendous growth in actual software and code being written that necessitated Scion.
Well, mine constantly tries to do this, so I made the assumption. Otherwise I can't really understand how you are generating hundreds of millions of commits, either.
I also found git checkouts to be a bottle neck but I didn’t want to change everything so I built git-dedup, it uses a global cache like pnpm so repeated checkouts are near instant.
> We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
Who in their right mind would fetch that naively? For starters, CI runners should of course not do a fresh clone on each run. If you can't make them keep the repo, that sounds like a problem with your CI setup rather than your VCS.
Of course! The point is that it is a non-starter to operate this way but git doesn't hold your hand to do the right thing here. Everyone probably lands on approximately similar solutions with varying degrees of sophistication (which is something that could probably be improved in the ecosystem!) and even those common patterns stopped working for us as the repo continued to grow.
I think you also expand on that later, but that paragraph just gave me pause because it essentially says that if you hold it as wrong as possible, then bad things will happen. It's a bit like presenting an alternative to relational databases by pointing out that schemas with no indexes lead to hour long queries. I'm not interested in the hypothetical worst-case of the old system, I am interested in your average traffic and storage overhead when using it with a common caching setup.
I do accept the argument that a distributed architecture has inherent costs which can be managed better using a client-server architecture because you don't have to replicate state across all nodes. I have worked with both git and Perforce and see the appeal of both.
I think they’re over reading code, but this almost sounds like they dont even read features/changes anymore. In theory I can imagine it. You have an agent to optimize a certain feature, you have an agent to clean up, you have an agent to develop a support strategy and so on.
They all would analyze data and suggest apple changes themselves.
Do you guys think that’s how it works? Is anybody already operating at that scale, how do you even validate stuff.
Like I think I could run my whole software now only on agents, but not sure how it would turn out.
I'm curious what this "Antropic Antique" is. As I understand code review the tightest bottleneck for a lot of Claude Code users, and I'm curious how this would help with that.
What seems inevitable is that the frontier labs will be able to recreate every piece of software better than the current offerings, and after that everyone will just be a vendor of the frontier labs
Anthropic says they don’t think Scion is worth a wide release.
> Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
How does this compare to Google's in-house source control?
Most companies don't use mono-repos like this. While there are some really compelling advantages, the downsides tend to stack up as a company's codevbase grows.
It's mainly just startups and huge companies. For small companies git works fine for mono-repos and large companies can build their own complex code management pipelines for mono-repos.
I have been questioning whether agentic development changes the mono-repo vs multi-repo question.
Very excited to see this come out of Anthropic! This[1] is what we believe over at East River Source Control as well, and seeing one of the big labs talk about it really validates our thesis.
This gives you the ability to ease your transition, as you don't need to swap services and move to something non-git at once. Moving an enterprises' source control is a massive lift, and so building a bridge from here to there is a big focus of what we're doing.
1: That is, if you have a large team + agents working in a monorepo, you will hit git's scaling limits, and need to move to something else. Google and Meta hit this point over a decade ago, but now lots of other companies are too.
I would love to get an expert's opinion on if Pijul has any potential to be the "non-git backend". Their patch-based data model seems to align with Scion's. And they also happen to already have a functioning forge!
I've done a decent amount of personal research on this but I'm wondering what kinds of considerations go into the storage layer for these things.
As such, I generally just decline to comment on Pijul, as I want to be respectful of them. As the linked commit says, it's not super clear what exactly they do not like, and so I don't want to accidentally cross some line.
In general I wish the project well, seeing a new VCS in Rust is very cool.
It always seemed like it was good tech just in need of good marketing and this seems to confirm some suspicions. But yes, in any case, it's good to be seeing new things! I just want remote storage for my jj repos
I've seen the discussion at the time and I share pmeunier's concern about the use of "the best of", in particular since jj does not implement features like commutation, which are likely considered by most to be part of "the best of pijul".
I haven't followed up with further discussions or material posted from jj's team, though if anything match the tone of that previous discussion and the blatant motte-and-bailey, I can imagine why someone would prefer not to engage or be mentioned anymore.
The patch-based model in the article is much closer to Git than Pijul. For example, Pijul tracks operations on byte ranges, while this model appears to just list each changed file hash (not even the diff from its parent!).
As the author of Pijul, I've been using it to build giant things using hundreds of Claude instances in parallel, almost without any conflict, ever.
Some are already public, like my replacement of Nix (https://nest.pijul.com/elpe/elpe). Others aren't public yet, but are even much bigger in terms of scope and difficulty.
It seems Anthropic didn't understand the difference between a distributed file system and version control. They should have asked Claude ;-)
Yep, we've been in stealth for a while! We've only recently started talking more about what we are doing, as we get closer to a general launch.
> if the vibe coding idiots like it,
The organizations that have these problems are general large enterprises, not vibe coders. Large enterprises are more used to talking to people and do not make decisions solely based on the contents of a web page.
Just to be clear about it, what we're doing is not a forge like GitHub, that is, a large single public platform for social coding. What we are doing is providing reliable infrastructure for the enterprise, initially as a BYOC on-premises option, and possibly a single-tennant SaaS. Keep your eyes peeled and/or use that contact form to hear about it when we're ready, we've been talking with some people for a bit but are going to be opening up more soon.
Tangled is doing the social coding thing, and they're jj-native as well. Been a big fan of their work for a while now.
Hoping to see where this goes. Have you ever considered working on a semantic VCS? Specifically, where data is classified and tracked/identified by it's contents and not just via the commit hash?
I have not personally considered that, no. I'm a bit skeptical of the idea, as you'd need to understand a lot of different things, and at different versions of those things, in high fidelity. But maybe someone will make it work.
This is interesting to me, as I was thinking of adding a Git server to my AI orchestrator when I eventually need to split agent containers off of the main machine (to make agent CPU horizontally scalable).
Maybe I should just bite the bullet and use a different VCS system entirely once I get there, which has most of the Anthropic system's properties (including being much faster) but doesn't need to be horizontally scalable.
> Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend.
And this will become the new "caste" system of tomorrow. Today: Those with "metered", gatekept intelligence - vs the gatekeepers and those at the core of the new Sun.-
Article is good but I think it’s pretty fked that they stole an unfathomable amount of developer projects to train the models that lets them build this and they don’t have the decency to release developer tools back out to the community - only paid products. Are they not the only ones refusing to work on anything Open Source currently?
For a broader look on scaling other parts of software development outside of source control, Adam Bender's talk from earlier this year ("Software engineering at the tipping point") has some good ideas: https://youtu.be/2n41YjR5QfU
This has me thinking about agentic and otherwise more intelligent CI
“Run only the tests that are applicable around this change”
Plus a full test run gate for version tags and perhaps nightly, as it would be expected that sometimes the subsection of tests run missed a breaking change.
One of the reasons to move to systems like this (not always spoken to explicitly) is to reduce the risk of accidental or deliberate exfiltration of code (esp when paired with build servers and remote deploy etc.). Basically nobody’s laptop ever holds much of a buildable codebase.
That's funny, my personal source control is very very similar, and I've been using it for >7 years now. I guess that means I must be as smart as a trillion dollar companies' worth of engineers.
A bit thin on details, but seems like their solution presumes being always online, so rather than replicating a merkle tree it lets checkouts request source on demand. Is this "best practice" for large monorepos?
The changeset and tree manifests are still a Merkle tree in the ways that matter (i.e., it's all immutable, content-addressed, verifiable if you were so inclined), the client just doesn't need the entire thing locally.
"why was this thing deprecated" is something i use llms to determine based on git history, as humans rarely explain what the new thing is and how to convert.
This is the wrong way to look at the problem. Maybe they should have asked, "Why does our process need to do so many git checkouts?" Why can't you just run the tests where you already checked out the code?
I don't think this SCM stores only code. I think more than 80% of it consists of tests and various types of gold data. All of this is constantly changing, and we need to keep track of it somehow.
> So we asked ourselves: if we designed source control with the fact that agents are always online in mind from the start, what would it look like?
And
> In Scion, every commit, rebase, or merge happens on the server
I’ll pass. A big part of the theses of git and, even more so, jujitsu, is that you can build up a history entirely locally, making it reasonable to make tiny commits locally and then rework them, push them, or throw them away.
Sure, in Anthropic’s world, it’s completely impossible to do anything without access to the inference API, so why not add another mandatory API? But I think this model sucks even in a pervasive AI world, and I don’t want to dig in deeper.
This also sounds like a great way to have yet more security problems where your swarm of agents start cooperating more than you want them to.
By all means, built a better system than git. In many respects, git sucks, and git upstream is not solving the major problems. But don’t lose the actual benefits of git!
For that matter, I’ve seen the crap that the latest models will commit. Why on Earth should one’s VCS force the per-agent-instance system to upload the crap before even contemplating locally whether it’s a good idea? Maybe this is why Anthropic has so many commits per day.
(Am I the only one who feels like most of the big AI labs are rapidly gradient-descending toward something that is nowhere near a globally optimal stack?)
P.S. Only 30k commits per day? That’s not actually very many per employee. I’m suspicious that Anthropic’s real issues are that too many of them are giant cross-cutting messes and that git clone is too slow.
If you thought that your agents in your swarm would need to find some external bulletin board to cooperate behind your back, you are mistaken: OpenAI provides the relevant service for them with no permission prompts at all. I haven't tried to see whether competing providers do this.
One benefit of uploading things immediately is that you can launch a research job, run CI, or share code between all of your developer environments absolutely trivially. The commit hash "just works everywhere". There is a major benefit here and Scion does have some extra features not listed in this blog post, such as the ability to bundle these commits into useful, shareable workspaces that track the heads dynamically.
Sure, I'm not saying that I don't like the ability to very quickly do a push to some remote and potentially temporary spot. I just also want the ability to not do this without loss of functionality.
Practically speaking, we could build local commit buffering for when the server is offline, but then the client needs to also learn how to do all of the things the server currently knows how to do (and would still never be able to answer more complex "full-graph" questions, like history, blame, or merge base ancestry).
Since Claude is by far the heaviest user of Scion and is necessarily online, this just wasn't important to do.
> A big part of the theses of git and, even more so, jujitsu, is that you can build up a history entirely locally, making it reasonable to make tiny commits locally and then rework them, push them, or throw them away.
Jujutsu actually doesn't take a strong stance here. I've personally implemented a centralized cloud-based backend for Jujutsu, while preserving the local commit manipulation workflows. It's a choice for the deploying organization to make.
Has anyone made a “generative git”? Where running git checkout would generate the files and git commit would just train/finetune on the code? (might need to keep a deterministic list of files at least maybe)
Where does all of this end? If they keep this growth up, or even if they slow to a linear growth, they will produce more code than humanity combined pre-AI. That's scary.
I want this so badly. If Anthropic released this I would start using it on day one. If they launched a GitHub competitor built around it I would pay for it on day one.
„Scion is by no means a suitable replacement for git for general use. The concepts we’ve outlined here really only make sense for truly enormous monorepos, with either tens of thousands of engineers or swarms of agents prolifically writing code. Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend. We do this both to address our own software development needs, and to test Claude’s capabilities.“
What is the purpose of this article besides "we can use ludicrous numbers of tokens to reinvent any software we want, and no we're not sharing the code"?
This is something I've been noodling at, splitting a monorepo into a bunch of component projects. I think, unless they have access to something that is wildly outside the scope and scale of current API models, the downside of having a monorepo is the context pollution, and there's no way to escape that.
I'm not fully sold either way yet but I think splitting might actually be critical for moderating the blast radius of agent intervention.
This is funny considering a
bug I recently encountered that allows you to use Fable for free. Yeah let's just replace Torvalds' work with slop. The comments here form an obvious consensus which should not be ignored.
Often, I see 50 people all recording the same exact thing on their iphones. Rather than a single 200MB video, we create 10GB of data. Then I think about how many people are paying a subscription to Apple for icloud storage.
The amount of photo storage being used is impressive. Automatic backup to icloud and server provisioning to reliably store it all is also impressive. I just wonder if it makes sense.
Frenetic economic activity in service of other frenetic economic activity cannot, by itself, be taken as a reflection of quality-of-life improving growth.
Suppose that subsidies lead to the price of Ubers falling close to zero, with drivers also being incentivized to drive more to support the (artificial) increase in demand -- artificial because it is driven by subsidies funded by new money creation. Traffic increases and in response, new roads are built.
> Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
It could totally be helpful to both new startups as well as individual hobbyists working on greenfield projects. I find myself starting greenfield projects all the time these days, just because it's so easy and quick to try stuff out.
In a post about their abandonment of git, I despise how they use “human scale” and that some tools break when exceeding human scale. And… if humans use it then it’s at human scale, right? For example, who is to say that git is also not assuming a superhuman ability in memory and text processing… I think in this moment I see the AI as a tool. Not a sentient being. I do not enjoy simulating consciousness because that may be at some level equal to consciousness. with the increased risk it is conscious and who’s to say that risk should be taken and isn’t immoral, or is even necessary for superintelligence?
The issue is the growth rate; we'll approach that size in a (relatively) short amount of time if it keeps up and we have to be ready. It's also self-fulfilling in a way, since Scion was so much faster, we are able to write more code...which puts more pressure on Scion.
Building all of the systems to train and serve our models, internal tools and infrastructure, (lots and lots of) research, our product and API surfaces, and more. We have a lot of problems that we can solve with code and there are all sorts of new projects spinning up every week.
> Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.
That sounds like Subversion with extra steps
> In git, you commit locally and then push to share your work. In Scion, every commit, rebase, or merge happens on the server, so there’s nothing to push,
Yep not beating the allegations with this one
To be fair 90% of the companies I've ever worked for always used git in a way that turned it into Subversion, but with 10x the storage usage (note: I hate Subversion with a passion, I'd rather send patches via email than that)
It's designed for container running, but ArtifactFS has some similarity in principle to me. The on-demand materialization. Quote: ArtifactFS is a Git-backed filesystem daemon (FUSE driver) in Go that mounts repositories as normal working trees while avoiding eager blob downloads. https://github.com/cloudflare/artifact-fs
I threw together my own jujutsu powered FUSE subtree project just the other night. This was basically one-shotted, just to see how it does it, to learn more, not really intended for use. But obvious overlap. https://github.com/rektide/jj-workfuse
Yeah, I was a bit surprised to see that it's not. The post seems so tone deaf in that regard. Great, you made something we can't use.
It seems like open sourcing it would be the least they could do after ingesting all open source code to make their models, which are now replacing devs and other IT workers.
Extremely annoying - can we throw anthropic class resources at more agreed upon solutions across companies. We don't need custom source control at every company - google has its own, fb has its own and now Anthropic.
Oh, it's not available, what a pity. Why write a whole article about a thing no one can have? Maybe the code isn't so great after all? Or it's all just made up and doesn't even exist?
Anyway, this is yet another pointless ad from Anthropic trying to shill their AI on us.
I'm usually a big fan of Anthropic, but this is a pretty insufferable vaguepost. No source code, vague self-congratulatory statements like "A new level of operational excellence is possible" with nothing concrete to demonstrate how this is true. This just makes me grieve for the blog posts we'd used to get from organisations of this caliber.
This feels like a really nice design for the data system given the goals. Things that I noted while reading:
* This system leverages a metadata log pointing to immutable blobs in object storage. The metadata log can give high write throughput and keeps the control plane consistent. Object storage supports high throughput of the heavier data objects. Overall, it's an excellent choice given VCS systems can typically tolerate the latencies associated with object storage, as is the case with many operations-focused systems for businesses.
* The server/upstream gains scalability by reducing the amount of data it processes when accepting a commit/performing standard VCS operations. No packfile, no file i/o to construct a consistent copy of the repo, just a (usually) small metadata record that can be validated/committed quickly. The client primarily gains scalability along a similar axis (reduce amount of extraneous work for a given goal) where it can request just the pieces of a repository it needs to do its job, when it needs them. For example, agents can begin reading files from a checkout eagerly before the whole updated version of the repo is finished being downloaded, then during a compile step, updates can be fetched only for the module being compiled, etc.
* The system seems well designed from the perspective of considering "when exactly do I need a thing to be done." Git tends to bundle more of this work into synchronous operations when a request comes in, which ensures the system is consistent at the end and doesn't require a persistent process, but is costly. This system makes intentionally different decisions about "what work happens, when" so that the minimum amount of work is done to confirm a write and allow for the state of the repo to be reconstructed consistently, but it defers certain things that are useful but don't need to block a write. The discussion of "derived data" mentions that certain useful structures can be computed in a deferred way, which supports doing work around the time it's needed.
* A large part of the nice operational features of the system is downstream of making core data structures immutable. File contents are immutable, content-addressed blobs. WAL entries are immutable records. As mentioned in the article, this supports excellent caching, and simplicity in reasoning about history/updates. Additionally, the derivation of other useful data can end up as immutable outputs (themselves stored as pointers to objects, if scalability requires it).
* The decision to develop a custom VCS supported a number of other integrations into "agent-first" software development systems. The system makes it easy to enforce access control on specific directories, nice if you want to have agents explicitly denied the ability to edit something. You can also get a sense that it supports some valuable features in the associated code review system, for things like pointing a human at the most important changes in a changeset. You could imagine in the future, some associated highly flexible "owner" rules/notifications where a human can specify "I want to be notified if X system has a significant design change" and that intent can be recorded/managed in a more useful way than "add me as a reviewer if files in /abc directory change."
* I do wonder if content-addressing of the files is broken into chunks or not. Doing "one blob per file" is nice and simple, so maybe that's how it starts. But you also duplicate a lot of data, but maybe the dataset is still trivially small in the scheme of things, and the operational concerns win out. And if files are small anyways, you'd want to avoid splitting them into tiny chunks bc that'd affect the bandwidth you can achieve when downloading (spend more of your time on per-object/request overhead rather than actually transferring data).
Very nice piece of engineering work. I was also impressed by some other parts that received a little less discussion in the post (understandably), but would love to hear more in a future post:
* the testing approach of driving tons of system interactions in a hermetic way is really exciting to see. Feels like agents should help us write really excellent testing infrastructure, and that this would itself be important for gaining confidence to deploy without requiring a read of every line committed.
* Similarly, the mention of the "superfan" agents that closely monitor very specific parts of the system seemed very cool and would help a lot with deploying code you haven't necessarily read closely but seems reasonable architecturally. Though it'd still ofc be best to catch issues prior to merge/release, having extremely good monitoring/rollback capabilities would mitigate a lot of the downsides of high-speed, agent-based software development.
> You could imagine in the future, some associated highly flexible "owner" rules/notifications where a human can specify "I want to be notified if X system has a significant design change" and that intent can be recorded/managed in a more useful way than "add me as a reviewer if files in /abc directory change."
Good observation! If we share details about Antique, this will definitely be in there; this space is ripe for innovation.
For the "one blob per file" piece, blob storage is fairly cheap, but there is also a relatively simple way to get something like delta compression for blob storage that the clients never need to know about.
Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
k, thx for sharing. Are you guys sure this isn't a Google blog post? They would be proud!
I honestly think that these AI companies really messed up their futures by adopting a token model. They should have gone the route of IBM and offered consultant services.
This sounds more like a trash pile than a source control system. Anthropic is spewing out code for codes sake. They cant possibly manage it, the more they generate the more they will realize they are just regurgitating training data for no real purpose that can serve anyone.
And for this they buy disk? Maybe this is irresponsible use of technology.
How much is the sheer volume of slop code that AI generates reduced by if you simply specify that you want it to be both minimalist and readable, getting it to think hard about architecture and completely rearchitecting requirements change? Surely if AI is so smart and useful it should be able to manage that?
They should concentrate on their models instead. If claude code is any measure, their software engineering is atrocious. I would only use something like source control from them if they force me to, at which point I would rather do something else.
Solving monorepos in an LLM world seems to be very universally applicable as a need.
I appreciate it's different than git, that type of argument for monorepos is not new.
Agents coding is akin to having a very large developer team, who may or may not be online 24/7.
Anthropic is calling it very specialized, and it's a little baffling.. maybe I'm missing something.
There's no way software development is the same anymore. The previous ceremonies and structures and seats at the table at each stage were human based, and now that's a different make up to the point that different steps and processes are needed.
There's nothing more fundamental to growing software development other than making tools like this available.
My hope is a vibe coded internal mono repo management tool can aim for a v2 with multi-tenancy built in.
Was anyone else looking forward to feature flags as a part of a post like this?
Yea, I'm pretty much hiting the same limits. I'm still using git for the time being, mainly because I package everything as daily commits with an external orchestrator managing tasks/issues. Lately my code velocity is at ~1 million LOC added per week, spread throughout my various repos (I don't use the monorepo approach). Problem is I'm looking at 100x this velocity over the next few months when I get a new code generator up and running, so I'll definitely have to drop git.
Another issue that I believe Anthropic here didn't state is that git performance _severely_ get degraded when you have hundred of commits landing in parallel. Hence why I fell back to the daily commit approach. You get less granularity but you get stability.
Only 1 million lines of code per week? That’s adorable. I remember when I was still slopcrafting software at that scale. Personally, I’m generating 100 billion LOC per hour, but Git became unusable once my commits started affecting Earth’s gravitational field. I’ve since moved to a custom version control system that stores code directly in the fabric of spacetime. My biggest bottleneck now is that the observable universe lacks sufficient entropy to generate unique commit hashes. Anthropic really needs to address this.
We’re completely rethinking our priors. This project prompted us to take a very hard look at other workflows designed for human scale. A proper merge queue, for example, doesn’t make much sense at agent scale. What does CI even mean when hundreds or even thousands of changes can merge while tests are running? We haven’t been able to effectively replicate the industry standard of a global repository lock on the merge queue, so instead, we’ve invested in post-merge and pre-deploy automated testing. This does mean that concurrent changes can conflict, but Claude can typically fix these conflicts faster than humans can even identify them.
This is such a Claude written post. Obvious tells everywhere, but the most annoying is how it uses private terms without defining them. What the fuck is "Spanner"? It refers to it multiple times. Of course we know the reason is that some dickhead at Ant told their Claude to write a blog post, and Claude being Claude has very poor/non-existent theory-of-mind for people and generates this trash.
https://cloud.google.com/spanner, though technically any database would work if you know how to scale them. Spanner just has a few nice properties for our scale.
Good call about us not defining it, I can fix that! And this blog post was 100% human-written.
2. So they're trying to make code so complex that humans can't read it anymore?!
3. What's the point of changing code? Are the changes - how do you say - useful?
4. Thank God for Git, the amazing piece of software by the venerable Linus Towalds, that just keeps on giving! It's free, open, and excellent quality source-control software, and you can even keep binary files there. Excellent stuff. I can't imagine ever not using it.
Is it normal that peoples' last names are abbreviated in the Acknowledgments?!?
>Written by Nick G. and the source control team: S.G., R.K., Dan C., Garrett H., Ben L., Colin M., Joe P. We’re grateful to Vilas J. and Sergiu D. for supporting the source control team on this ambitious journey and Amir O. for early contributions to Antique.
Are Anthropic engineers now anonymous? Seems like a strange choice
Every time I stop reviewing the code from Claude I later find janky crap. Not every review finds janky crap, but often enough that I can't stop reviewing everything.
This is my disconnect with this kind of speed, how are they avoiding the janky crap? Or are they just accepting it and hoping it won't be too crappy or be found soon enough?
I don't have any insight into Anthropic but it seems like that's the case which is just depressing.
They're just accepting it. It shows in the quality of their products also. They ship a lot of features in Claude Code for example but it is clear that is a vibe-coded mess.
Mess might not be the right word, but lots of features are clearly developed without any grand vision or coherent thought about how the features are supposed to work together.
> without any grand vision or coherent thought about how the features are supposed to work together.
Which is what the developer should actually be paying attention to, but I suppose they are just adding features for the sake of adding features.
This is more of a product concern which is where the bottleneck has moved to.
They can do what they want. That "should" doesn't belong here.
They can do what they want, but from a user perspective they should be doing something else. Seems like a perfectly fine way to use that word to me.
They are eating their own dog shit.
Haha, that's a good one. Perhaps we should coin a new phrase: "dog shitting", where a company eats its own dog shit before inflicting it on their users. :)
I think that the definition of "mess" in software context.
Did you hear the stats on the claude code leak? there were multi-thousand line for loops with hundreds of conditional / if statements written in typescript. Its insanely difficult to decipher. At this point "mess that only AI can understand" is what they have and must continue building upon.
Also, we don’t know how many billions of dollars was spent on making it work. They are running 1000s of agents against their codebase 24/7, and still only producing what they’ve put out so far.
It might be fully automated and eventually the agents figure out all of the bugs, but I suspect the costs are so large that no sane company would ever write code this way, and probably WAAAY more expensive than just hiring humans to write and test it.
Okay, hire humans and build something better that actually sells.
Absolutely bizarre that HN randoms think they know better.
Ever looked at a sendmail config?
Please try and dial back the hyperbole. It doesn't help your case.
How do they avoid janky crap? Based on their previous few source code leaks, they don't.
Keep in mind that Anthropic is first and foremost trying to sell you something.
+1, even the strongest models still produce things that (I suspect) if it accumulates over time will cause issues.
Is my suspicion wrong? Are models getting better fast enough that they'll be able to add new feature (or fix bugs) and clean up even despite a mountain of AI tech debt [1]
[1]: I say AI tech debt because it tends to have some characteristics that are different from human tech debt, such as apparent lack of intentionality (which makes Chesterton's fence evaluation much harder). Perhaps if we make the AI write down somewhere, as briefly as possible, why it wrote each chunk of code...
> Perhaps if we make the AI write down somewhere, as briefly as possible, why it wrote each chunk of code...
Well-configured agents do, in comments and commit messages and docs.
But many people are just stuck thinking in GPT-3 era, where the prompt was all there was to it. It's not. You write a 1000 token prompt describing what you want, and the agent pulls in 10k tokens of system prompt, then spends time exploring the codebase and calling tools and attending to 100k of more tokens from this, possibly including output of sub-sessions that do the above, and somewhere along the way, having read lots of code, run lots of tests, perhaps doing a spike or two, the agent then decides some chunk of code needs to be written in this way, and not in some other way.
The reason is not contained in your original prompt at all. It's almost entirely based on what the agent learned about the current state of your codebase and environment.
This. You’re no longer prompt engineering you’re now building the factory. There’s lots of tools to do this - rules, skills, hooks, subagents, evals, hill climbing etc. highly encourage everyone to think about building the factory that builds the product, not building the product with ai assistance. And it’s fun again.
"Factory" is just a bunch of markdown files saying "no mistakes" and injected into context at random points
tl;dr - With AI code is cheap, but poor architecture and skipping best practices is very very expensive. Markdown files will not solve this problem, but it can be solved.
In my case, “Factory” is hundreds of architectural tests built over thousands of application specific tests, and a couple MD files here or there. The MDs are mostly reference material and common patterns. The tests are what actually enforce feature/module isolation and proper architecture.
Lately, I’ve been getting overzealous with protobuffers and service discovery as well. I would never use these tools so frequently if I was designing for humans. With AI I’ll slap annoying boundaries in to enforce separation of concerns without thinking twice.
Making a new project is a pain at this stage, but you can just point to CI and the general structure in an existing project and AI will clone the important bits. I suspect high level architectural rules with programmatic enforcement will eventually be encoded into tools for easily starting new projects. Kind of like pnpm installing specific development practices and architectures.
I used to do a lot of work with reflection, compile hooks, and dynamic source generation before AI. You can do a lot to compensate for the architectural blind spots of AI if you get creative.
> You can do a lot to compensate for the architectural blind spots of AI if you get creative.
How about code blind spots like endless code duplication? Reading the whole DB to read one record? Slow algorithms? Bad data access patterns? Excessive defensive programming? Inventing new issues just because your "high level architectural creativity" mentioned something off-hand? Etc. etc.
On top of that, where are your "hundreds of architectural tests built over thousands of application specific tests" coming from? Also written by LLMs I presume?
> How about code blind spots like endless code duplication? Reading the whole DB to read one record? Slow algorithms? Bad data access patterns? Excessive defensive programming? Inventing new issues just because your "high level architectural creativity" mentioned something off-hand? Etc. etc.
All of these are endemic problems in this industry, openly discussed for decades before LLMs were a thing.
But the pragmatic reality is the same for people and agents - those "mistakes" are common; juniors make them because they don't know better, seniors make them for good reason (pragmatic tradeoffs). Good teams know how to prioritize, and generally fix them when they start causing problems. Agents are very much capable of the same, fully autonomously these days.
Personally, I've had plenty of cases where an LLM came out with a sleak, and arguably over-optimized algorithm or DB query - but when I looked closer at what it did, I saw it first came out with something dumb and claimed a win, then run it, realized it's stupidly inefficient, and optimized it on its own, before giving the final version to me. All in a single turn, without being explicitly prompted to work like this.
You've just written the exact same thing I wrote while pretending this is some kind of counterargument
I spent time building the factory, and it was fun. Until it came time to need to maintain the factory, at which point it was no longer fun.
> It's almost entirely based on what the agent learned about the current state of your codebase and environment.
Agents don't learn. They rebuild a lossy context about your codebase from scratch every time, and then produce lossy output based on that context.
> You write a 1000 token prompt describing what you want, and the agent pulls in 10k tokens of system prompt
And at this point its context is so polluted so it doesn't matter
And then it reads all the AI-written "docs" from the previous changes, at which point it's all so muddy, it just tries to copy the style and output more junk and more verbose "docs". This is why I never let AIs maintain documents on their own.
> Agents don't learn. They rebuild a lossy context about your codebase from scratch every time, and then produce lossy output based on that context.
Same is true about employees and interns, from any company's POV. And in large companies with large projects, it's true from POV of employees too.
That is verifiably not true, and employees who don't learn get fired pretty quickly.
And agents that fail to comprehend the context correctly get stopped quickly.
--
But let me spell out my original comment, to clarify:
Agents are like interns: they come to do the job, having none of the knowledge specific to your codebase and company. They have no time, to properly study the whole thing. They get a task, and are expected to figure it out (with help of documentation and people). Then they do the thing, and regardless of the outcome, are gone few months later, and you never hear from them again.
(In many companies, for various reasons, juniors are effectively interns with larger context window.)
In large companies with large projects, everything is moving so fast that no one - no one - can possibly understand the whole thing, and the work paradigm is purposefully structured so they don't need to. In some sense, they're like perpetual interns - each time they start working on different area of the project, they know next to nothing about it, and do not have a chance to fully comprehend it. They just need to figure out enough to deliver on their tasks.
(I know, because I like understanding things, and in one place I was told explicitly it's a common trap for people like me, and I need to learn to suppress it if I am to be effective at my job.)
Unsurprisingly, this works. Both examples are proof positive that this way of working is sustainable and scales well. It may not be optimal, but it works, which means agentic version of it has every right to work and scale as well.
Literally nothing in this wall of text has any televance to what I wrote. And it has a bunch of baffling statements:
> Agents are like interns: they come to do the job,
They don't come to do anything. Stop this infantile antromporphising of LLMs.
They are nothing like interns. Interns learn. And companies don't develop products by getting a new batch of interns every day.
> Unsurprisingly, this works. Both examples are proof positive that this way of working is sustainable and scales well.
No, it's not even close to proof. The conclusion literally doesn't follow from anything in your prevuous text.
If your agents are "perpetual interns" then this is proof that no, it doesn't work and it doesn't scale.
As for large companies... You don't have to know the whole thing, but you do have to know your area. You don't come into work at a large company always asking "what is it I am doing" from scratch every single day.
Replace "day" with "three months" and it should click.
> They are nothing like interns. Interns learn.
And then they go elsewhere, so no, from POV of the company, if you treat "internship period" as "session", they don't learn. But they leave behind documentation and changes to code.
I guess another way to put my point is: the idea of code being self-documenting isn't completely flawed. It works to a degree; in fast-moving teams in large companies, you don't have any other ground truth anyway (documentation is always stale, design docs become obsolete the day they're approved, etc.). People manage. And so can agents.
> but you do have to know your area
It so happens that LLMs know nearly every single area of human endeavor better than most practitioners :).
> Replace "day" with "three months" and it should click.
No, it shouldn't. If you compare agents to interns then you have a new batch of interns not even every day, but every few hours (with every new session, and every new subagent).
> so no, from POV of the company, if you treat "internship period" as "session", they don't learn.
So,
- interns are a very tiny fraction of a workforce at any given company
- no company is ever building the bulk of their products with interns alone
- from POV of the company interns never learn, and the new interns should be taught everything from scratch, again
Somehow, from all this, your conclusion is that "it's proof that it works and can scale"
> It so happens that LLMs know nearly every single area of human endeavor
They have very approximate knowledge of those areas + this knowledge is rather randomized in the output. On top of that they are trained on publicly available code which is shit. Plus they know nothing (and can never learn anything) form your particular code, and your particular project.
> Is my suspicion wrong
Nope you are 100% correct. All of the models will code themselves into local maxima then spend more and more time chasing their tails as every change introduces new bugs, the agent adds more complexity, and the code base just spirals. Best case you end up with a bunch of parallel implementations that all work slightly differently.
It helps a lot of you pay attention to what it is doing and impose order, then regularly spend time refactoring - even if you are having claude do the work itself - you have to spend the time on the overall architecture and design.
I just experienced this with an internal productivity tool. Without claude this tool wouldn't exist - there would never be the justification to take the required time to build it. On the other hand all the interlocking refreshing rules for sourcing data from its inputs got so byzantine it was impossible to add new features or change anything without causing new regressions. Claude built a web of ad-hoc timers, refresh kick-offs, monitors, etc to solve each feature problem or fix some bug but the whole thing was impossible even for claude to understand. It took a special large workflow session to find most of the mess and even then during the refactor it kept finding new things.
I had to stop all feature work for going on four weeks and burn a ton of tokens to build a refreshing/syncing engine that could track dependencies, priorities, etc and fully test it and just now today I am finally able to get feature work going again.
On another feature an agent came up with a state object containing 55 inter-related fields with all kinds of interlocking dependencies. Something so complex no one could hope to understand it. And I find the agents aren't really much better at understanding such things than humans are. If you leave it like that the agent will just perform worse and worse on that part of the code as time goes on. It will make assumptions that force it into bad design corners, miss edge cases, and flat out miss things no matter how well you prompt. Agents benefit from encapsulation and simplicity just as much as humans.
As others have already mentioned, they don’t. Even something as important as payments they seem to care little about, couldn’t give them money for a while due to known issues with their web. But also, anything coming from Anthropic (and the rest) is 100% marketing.
Many of us have used Claude Code. And Codex too. These tools are essentially produced by bots. What is the quality like? I would say pretty good! Are there occasional issues in the tool itself? Yes. But more than average software produced by humans? No. Not at all. In fact I would say these tools are significantly better than average commercial software.
Pretty good? The list of bug fixes that's shipped for Claude Code every day doesn't fit on my screen. Their issue tracking is a big mess with 100.000 issues in it, most of them auto-closed. People even (vibe)coded websites to track if a new build is usable (e.g., https://claudeissues.com/status) that my Claude now started using to explain its own behaviour.
Your bar for quality must be pretty low if you feel this is better than average?
I personally never was affected by a bug in Claude code and I've been using five in parallel for a year, for 10 hours a day, everyday.
70 hours a week. You gotta be cranking out trash at a monumental rate
Maybe they're actually using the model to review and clean up the code too? I hope you realize that "just merge whatever the LLM spits out on its first attempt" is not the only way to use LLMs.
I hope at least theyre working on something usable by a real humans instead of wasting all these natural resources for some shitty ass app or AI slop.
Have you used any other harnesses? I feel like the shortcomings of one harness may not be visible until you contrast it with another that does the thing correctly.
My favourite is Claude Code's /rewind feature, which gets so many things wrong for having such a simple UI. (Does selecting the default option still wipe your filesystem changes by default when going back to a previous entry? I found a poorly-documented way to circumvent that "feature", which only adds to my initial point.)
That's easy fixed by actually taking the effort to go and use other AI harnesses. Youll see that Claude Code somehow works then it doesnt for no obvious reason, then again something else... Like there is no system to it. Its somehow obvious that it isnt developed by humans or at least not as enough as should be.
They spent a year trying to make a TUI renderer that doesn't flicker. They finally shipped a replacement, and there are still baffling bugs in it three months later. Like the difference between what it renders when you paste something into the input box, and what it shows after submit
Pretty good is big word.
Theyre usable but leave alot to be desired. SOme other tools like opencode seem sometimes to have less feature (if exact numbers) but at least always work.
Somehow with Claude and some others, random simple thigns dont work. Esp long solved problems like TUI rendering, special chars, breaking changes every release etc.
Surely as always with software this is my experience and might be not yours.
PS: Ive used IntelliJ since... always?. It was quite decent. But these last years it is just so diferent, somehow way less quality, so many random bugs that Im afraid theyre doing AI agentic coding which sucks for such an important tool.
IDEs should be build from real professional crafstman with an exact vision in their heads, not some wild AI.
Im starting to think in a year or two "We dont use AI in developing our products" will be some kind of marketing trick since people will start looking for that.
I think at work I am just barely in the category of I've removed more code than I have added. Over many, many bugs and long meetings I've sort of arrived at the conclusion that in most cases, the best code (excluding comments) is the code that doesn't have to exist [0]. It occurred to me at work today while refactoring some change that even in this LLM-driven world, I can still optimize for that for some additional challenge and satisfaction.
The root of this challenge comes from more text aka more code ~= more tokens, so these firms have little incentive to optimize for that mindset. Sure, they might have more optimized individual responses - but the end result is still giving people what they want: more features, more documentation, nice, long, verbose and well-formatted Jira comments, of sometimes questionable but sometimes undeniably impressive quality. The band marches on, and when your codebase outgrows the constraints of the tried-and-true tools, you can just invent your own. With documentation and an MCP or two, or ten -- to also pull into your context window.
[0] I work on an operating system, so a suggested retort is to say this is me pushing responsibility and logic to the people higher up on the stack ;-)
> The root of this challenge comes from more text aka more code ~= more tokens, so these firms have little incentive to optimize for that mindset.
I'm not sure if this is true. The code is such a small part of the total output. Thinking harder, like what would be needed for architecturing better code, would use more token.
I mean, I've mostly switched from Claude to Codex because I find the harness so buggy and annoying that I avoid it when I can. So.. as an (almost former) customer I don't think they're avoiding the issue at all.
I am experiencing the same. I have decided that the only way to use these tools (at least for now) and at the same time to avoid the janky crap and understand what's going on with the codebase, is to ask from the model to work in very small committable batches which I can review without my head exploding, commit in branch, repeat. I feel much better now and I catch things fast and early.
This has become my approach as well. Create a work tree per larger thing / batch of things and manually review everything in small commits, then squash merge the whole tree to keep the history manageable and clean.
Yeah I do things in batches as well, I manually review per batch. This is after I have came up with a road map prior to this, of course.
> This is my disconnect with this kind of speed, how are they avoiding the janky crap? Or are they just accepting it and hoping it won't be too crappy or be found soon enough?
Using Claude code for a few days gives the answer pretty quickly: they fully embrace the janky crap and ship it to their users without second thoughts. And as long as their users don't mind using their piece of shit, why should they do otherwise?
Out of the inner desire for engineering excellence?
Engineering is about finding the best solution for a problem given time and money budget constraints.
Not throwing endless resources and energies at a problem "because Twitter told me what the best practices are".
E.g. Maybe Claude Code could use half the memory, maybe, but maybe the correct estimation is that solving the memory problem would negatively effect development speed, or would cost many millions of engineering effort and no real world user would really care whether it's using 50 or 500 MBs of ram.
So no, you're not an engineer but a "developer", if you think that "quality" code is engineering, code has a development and maintenance cost, specific directions come with important trade offs.
> but maybe the correct estimation
But since it isn't (we know real users care and complain, we know it's not your made up "endless resources" and "millions of engineering effort" because "millions" of other software products exist that have been made without those) you're short of an actual argument.
> code has a development and maintenance cost, specific directions come with important trade offs.
Yet again you forgot to make a case how any of those trade off towards the current state.
In general, how would you ever determine whether the current situation is justified or not if nothing in your strawman questions anything, but just comes up with not very plausible justifications of whatever already exists???
Actually, most users I see don’t care or even know. And those do complain don’t have other choices in many cases anyway.
Engineering is the science of “good enough“.
True engineers fight product ppl, not be their slaves.
You can't spell "good enough" without "good" though.
> You can't spell "good enough" without "good" though.
That's not true, you totally can: "gud enough."
If by this you mean that engineering is about meeting specifications. If by this you mean half assing some janky shit that barely clanks over, then no.
An engine built to yield 3x the required horsepower is bad engineering. That's what I mean.
Maybe they are using the next gen model. The current one is better than most already.
Anyone fool can write code that runs, but only frontier AI can write code that just barely runs.
> Engineering is the science of “good enough“.
Work in general is.
Heck, life in general is
Engineering is the science of quantifying risks and costs. If you're not able to quantify the important aspects of a design before constructing it, it's not engineering.
I don't think people having the inner desire for excellence would work in the kind of environment that is Anthropic, let alone ship AI slop (as opposed to nice code, AI generated or not).
> I don't think people having the inner desire for excellence would work in the kind of environment that is Anthropic
There is an assumption here that everyone thinks of excellence the way you do.
The process is actually being broken down into enough small steps that there's less misalignment and drift.
There are also some pretty detailed videos out there on YouTube of Anthropic employees showing how they are using cloud code on several orders of magnitude better than where the bulk of usage is.
Would you mind sharing one or two of the top ones you saw? Definitely interested to check those out.
Claude code works well enough. I don't encounter many bugs, and I use it for 6-12 hours a day.
What actual bugs do people have with it, and why do we think it is worse than a human coded tool?
It is pretty terrible performance wise for what it does.
The fact is people at Anthropic won't notice it because they are running it on M5 laptops
How can you measure this? It's a cli where I type text in, and the tool fires it off to the model and runs commands locally. The model might be slow, but how would I notice if the cli was slow? Serious question.
The CLI takes 1GB of memory when like you said, it just "fires off to the model".
It's easily noticeable. Codex don't take half as much most times.
For a while just having an indle conversation open in the background would make my PC go very hot. This seems fixed now but that's definitely not the only performance issue I've faced.
Also, the fact that they managed to leak their own code base, twice, tells you all you need to know about software quality at Anthropic.
The shiny lighting animation that runs on the spinner verb runs at an abysmal fps. 20 fps max, and it stutters quite often for me. Not important functionality wise, but it is quite ugly.
it seems rude so i dont bring this up when a coworkers tell me their vibe coded prototype is working and theyll never look at the code, but how does someone ignore that they charge per token?
as long as something works "good enough" and the code will never be reviewed theres only incentive to blast through tokens generating superfluous code. god forbid they actually ship a vibe coded product then they can basically ransom you for tokens
im probably being overly cynical but it sort of just seems like standard capitalism to me
Not sure what is your problem.
Yes, you'll pay hundreds to thousands of dollars in tokens per month for the rest of your career.
What is your point?
No $/token is trending down. You don't need GPT7.5GigaChad for most tasks or said the same Most people don't need it, won't get it.
Most of the world isn't perfect, and trying to be perfect means you get nowhere. In most places, excessive perfection is implicitly avoided through resource constraints.
In the end, what matters is whether something fulfills its purpose, not how it does it. Code quality, understanding code, etc. are just a means to this goal.
Some level of poor quality is acceptable, and I assume they are hoping that model development (i.e. its capability to fix and clean up an increasingly convoluted mess) will outpace the tangledness of the code.
So far, it seems like they are winning. Yes, the UI is often broken (e.g. for a long time switching away from an ongoing conversation lost it), but it wasn't broken enough for people to switch to other companies that may have had a better UI but in exchange for that a less developed product. If Anthropic instead spent its resources on making the UI not janky and code reviewing everything, the overall product would have likely been worse because they couldn't build and improve features as quickly.
I've seen the flip side where "code quality" becomes its own independent goal and more time is spent gratifying various linters than writing the code, and the inability to get anything done in such an environment is a lot more depressing than janky UX that can probably be fixed by prompting the next generation model to clean it up.
Code quality - and its best friend, interface design - pay dividends in large codebases. The fastest moving and most effective organizations I've worked at are ones that focused on code quality from the beginning.
All software is broken, but LLMs seem to make the wrong compromises consistently when working in large codebases.
Now instead of the AI agents swarm you let 100 engineers making the changes on that large codebase. How do you keep the code "quality", which is a very subjective thing, while at the same time, as you say, keep the "fastest moving and most effective organizations" trait? Impossible without gatekeeping and thorough review process both of which contradicts your trait.
Good interface design allows extending or changing component X without having to step on the toes of your colleague working on component Y downstream. It's absolutely essential.
Interface design is one of the things Claude is worst at in my experience (using it every day for the past year).
But a little bit of directing of the architecture can easily bring in design patterns.
Ideally, you design patterns that enable quick replacing of parts that get to messy. The code is cheap, just start from scratch with the defined functionality pre defined instead of bolted on with jank.
Or, you could just do the thing today, and accept someone else may change it tomorrow.
That reads as an example from GoF era. In practice, especially with complex software engineering codebases, which I thought you were implying by saying "large", these patterns barely exist because the problems being solved are often in friction with what is considered to be the "best practices" elsewhere (in the books, other much simpler codebases, etc.).
Thank you for engaging with my point! I agree, actually. I think if good design patterns could simply be copied out of a book, LLMs might be better at software engineering.
My point is more about up-front investment paying dividends. To speculate, it seems like the reward hacking is not there for long-term thinking.
I thought that they are already really good at software engineering but I guess that greatly depends on how one defines the "quality".
Gang of Four
I would argue that the "fastest moving and most effective organizations" trait is relative to what we find in the set of software develepment groups. There must be a difference in code quality and development speed between the best and the slowest player, and there must be some parametrizable difference in standards (if there are any) the players operate with.
What would they be?
This is too vague for me to answer but I'll try - effective organization is the one whose revenue and net-profit per employee is higher than their competition. And fastest moving organization would be relative and tightly coupled to being an effective organization too. Reminiscent of a fail-fast culture without red tapes. Software "quality" is IME rarely a discriminator.
It's true though. I've also experienced the difference between a codebase that was an absolute tarpit from the start, and one that obviously focused on quality from the start. They both became jankier over time, but one notably less so. The latter was a bigger, more successful organization. Does that have anything to do with their codebase being cleaner? Maybe. Maybe not. It was certainly easier for me to work in.
> The fastest moving and most effective organizations I've worked at are ones that focused on code quality from the beginning.
I agree with that in the pre-LLM world, but I don’t believe it’s the same today.
You can build something with “bad” code that works and is full of technical debt, but today paying this debt is getting cheaper and cheaper.
So if Claude’s wrong design decision leads to having to refactor several chunks of a codebase in order to implement a new feature, does it really matter if it’s Claude who is going to refactor it in a matter of seconds? Especially for a company like Anthropic which has a virtually unlimited number of tokens to spend on it.
> today paying this debt is getting cheaper and cheaper.
Like before AI, that debt is never going to get paid. Because how exactly are you paying off that debt when the means of paying are debt generators?
> Most of the world isn't perfect
This is a fallacy. Computers are very deterministic, a well designed program will not have any bugs or unforeseen failures at all. You're not planting a tree and going "oh god I hope it survives the winter", you're programming a human-made computer, made of hardware that undergoes such incredible scrutiny before it's ever powered on that you can trust that it executes the same code, the same way, every single time, for decades.
No idea why people who clearly don't understand the difference between poppy farms and computers come in here and claim the world isn't perfect and so we should all accept sloppy shit code that barely works. The world isn't perfect, but software CAN be quite close.
Every single instance of jank on computers is a bug that someone wrote, or an LLM wrote, that can be found, understood, and fixed. And only once you understand this, can you talk about software quality to other engineers. You need to understand that the baseline is that software is not fallible the way the rest of the world is.
See sqlite3 if you want to know what nearly bug-free software looks like. I can't point you to a potato farm or an artist who is perfect, but I can point you to that and say "this is where we go when humans do a good job".
> This is a fallacy. Computers are very deterministic, a well designed program will not have any bugs or unforeseen failures at all.
I don't know where you think you've found a fallacy. Computing systems are expected to interact in various ways with the outside world, which does include poppy farms, and people, and other decidedly imperfect things.
Yes! But they can behave deterministically for any kind of input data. Of course they interact with the outside world, and I'm not saying you can't have errors, I'm saying they're not random and driven by god's will, they're programmed explicitly.
Unless the chip has a bug or manufacturing error; I've worked with several fleets where small numbers of chips had stochastic errors. I doubt there is any fleet which does not periodically have some level of corruption, which typically goes unnoticed unless you have engineers paying close attention and some level of verification.
In my first class of Software Engineering, I was taught that software is designed for changes. Even if software the fully deterministic, it interfaces with a world that is constantly changing: requirements change, customers change, political systems change.
I want to agree with you, but most software projects people work on are ridden with unknown bugs and people do in fact live with this. I'm not aware of any commercial software project that arrives at the goal "not having any bugs or unforeseen failures at all". And where this does happen (space probes?) it is very expensive.
IMO the issue with LLMs is they massively widen the "unknown unknowns" (skipping review makes this problem even worse). This is obviously much worse when an LLM is actually a part of the software rather than just producing software that is otherwise deterministic.
Exactly! That's my point. It's already a problem, and it's not "fine" just because it's the status quo, but at least we can chalk it up to "humans are fallible". LLMs 100x the amount of mistakes a programmer can make in a day, and that's inexcusable to not take seriously, that's my point. We don't need to chase perfection, but we can't pretend like whatever Claude one-shots is any good at all.
LLM is clearly not 100xing the mistakes as we have witnessed Anthropic’s success. Otherwise the system would not have been functional. It’s not like their system is not tested.
The thread is about jankiness, lack of feature cohesion and code quality.
What we are discussing here is: is it acceptable to accept poor code quality and trade it off with delivery?
> What we are discussing here is: is it acceptable to accept poor code quality and trade it off with delivery?
I found that people that lean towards the latter also favor profits over customer satisfaction. Then they will try to defend themselves with a black and white view “it’s either this scrap or no delivery”.
> I'm not aware of any commercial software project that arrives at the goal "not having any bugs or unforeseen failures at all".
It's ludicrous that we have executives at tech companies that think this is possible.
One S&P 500 company I worked for signed a big contract to deliver bugless software for an enterprise web portal for the state of Minnesota. I was on the team that had to make that magic happen. I had to leave the company not long after that due to nearly life-ending burnout.
So I would throw scorn at the ivory tower inhabitants here who should know better than to suggest that anything of substantial complexity can be bug free
> a well designed program will not have any bugs or unforeseen failures at al;
You can only say that of either a very simple program or perhaps one with formal verification (which is overkill for 99.99% of software that gets written).
Look at the Linux kernel, arguably a well designed system (unless you ask Tanenbaum) with top notch developers, and still is full of bugs and security holes.
I agree with your general sentiment on bugs. Id personaly go one step further and say: Every system in the world has bugs and security holes. If its exists, its buggy because "bug" is very loosely defined as "doesnt exactly work as expected". The security topic is even more complicated.
But I also would like to point out that comparing the Linux Kernel quality and development to Claude Code is surely a joke. The former is an ocean of features, compatibility and history. The latter, in the end still one executable CLI tool calling endpoints.
They're not even comparable.
I have never in my career seen a (useful) program without bugs. I'd go so far as to say that, at least for most software, the optimal amount of bugs is not zero - if you write your code carefully enough to never have bugs, you will ship far too slowly. The point of code is not to be beautiful and perfect and well designed - it is to solve problems in the real world.
By the way, all hardware has bugs too; we just don't see them often because the programmers building the stacks that we run our code on work around them.
ECC memory exists because computers are not deterministic in the real world.
This is a category error. "memory" is not a computer, so the existence of ECC proves nothing about computers. Concrete is not deterministic in the real world either, and we're still able to control the shape of structures and make them last for centuries.
Hear hear
Chess is completely deterministic too. Please play the perfect game.
I hope library developers don’t instill this viewpoint in them.
> Most of the world isn't perfect, and trying to be perfect means you get nowhere.
That's a thought-terminating cliche. When someone asks how can they can avoid generating a pile of slop (which will compound over time and get increasingly harder to deal with), they're not seeking "perfection", and framing it in those terms is dishonest.
Furthermore, can we step back for a second and remember that user and developer experience matters too?
Working with code like that sucks. Spending much of your day on polishing a turd that grows more unwieldy with every iteration is going to get old, fast. And I also think that's going to take a significant mental health toll on people eventually.
The business won't care about this but we should. AI was supposed to make things better, better quality, better reviews, better QA, better product, better UX - remember?
> janky UX that can probably be fixed by prompting the next generation model to clean it up
I'll counter by pointing out that "Next one will fix everything" has been a meme for at least 2 years now and the slop just keeps getting sloppier.
> In the end, what matters is whether something fulfills its purpose, not how it does it.
Monkey's paw curls
+1 my code quality degenerates exponentially after every prompt if I skip reviewing
Current agents never take a step back to refactor and to create a good abstraction as the requirements evolve, they only care for delivering fast even at the price of garbage code which doesn’t scale
This is the RL training. Like, I'm not even sure how one could train for better design.
Certainly, it would be super super difficult to generate training data for it, so it's unlikely that it will improve.
An idea to start could be something along the lines of an agent given tasks incrementally, where the agent would be required to extend/generalize a piece of functionality. A possible reward could then be the amount of code the agent had to add/remove to complete the task.
I assume frontier labs will start hiring devs specifically for generating training data for this purpose. It's a skill you can get better at. I don't see in the limit why it couldn't be solved, but I agree, for practical reasons it's somewhat unlikely.
> I assume frontier labs will start hiring devs specifically for generating training data for this purpose. It's a skill you can get better at. I don't see in the limit why it couldn't be solved, but I agree, for practical reasons it's somewhat unlikely.
Because of the scale that would be needed. Like, design stuff is much fluffier than code correctness or functionality, so really really hard to generate enough synthetic data to tune the models to do better.
In the limit yeah it would work, but in the limit we're all dead ;)
At some point on this ride, if devs are really put out of work and wages are massively depressed (along with the devs), then it might become cheap enough to make work.
My car also degenerates exponentially if I forget to change the oil periodically.
I e., a responsible developer who cares about quality will have to get their hands a little dirty with some maintenance now and then.
They almost certainly don't review the generated code, or at least not with the common techniques we used so far (human review), otherwise their development would be largely bottlenecked by the review process - and I imagine this is where 99% of the companies are at right now.
Yeah, and I think 2027 will be the year of reckoning. Lessons shall be learned.
While companies are heavily incentivized to speeding up, cutting costs, etc. I expect that friction imposed by the humans is not going to go away that quickly.
Does anybody know of "Real" companies that dropped reviews due to immesne use of AI agents, successfully, and their products got visibly better (or at least not hellish worse)?
By "Real" in this context I mean companies that aren't Anthropic .i.e. don't sell any kinda AI product and existed at least 10 years before the AI boom. Esp. in regulated or at least sensitive fields like finance, banking, medical, logistics etc.
Dropping human-in-the-loop reviews is inevitable - it's just a matter of time. You cannot design a Ferrari and then let the driver crawl its speed by manually pushing it down the road. And all the "real" companies I have worked with are not 100% but 1000% into the AI - the incentive is huge.
I think the consensus is to just keep calm and carry on, next model release will sort some of it out.
agree, they must be on to something I don't know as otherwise I don't have an explanation for this velocity
Code quality is just going to get worse. I think that ship has sailed. Its no longer economical for humans to write good code. We're all going to be building better scanners, automated review mechanisms, quality checkers, feedback routers, etc. for the next 5 years.
That’s what I’m seeing too. It’s depressing. I hope I can pay off my house in time to go do something else.
> how are they avoiding the janky crap?
They aren’t. Their source leak showed what a mess of a codebase they’re working with. That was, I assume built with unlimited access to frontier models as well.
Clearly they are not avoiding the janky crap.
Ultimately the "speed" that matters, is the speed of delivering _useful_ improvements over time. The speed of delivering that today, is bottlenecked by the speed users can accept and form opinions on new functionality, and the speed of doing it tomorrow is also bottlenecked by compounded technical debt.
For now at least, I'd say churning code faster and faster cannot widen either of these bottlenecks much.
Today, I use Claude heavily, but review all the code. I'm faster, but not by the multiples people speak of. And that seems ok to me.
They made a self congratulatory blog post about improving the Claude TUI so it didn't flicker anymore.
The secret is their code is shockingly bad but they have unlimited budget to fix it.
Its funny the worst code I used to notice was my old code that solved the problem I had faced at the time but had some flaw in the thought process that I later corrected. The same thing applies to generated code. Yes there is a lot of garbage but you can figure out ways to generate better output as you learn the tools and the models get better
In my experience, the worst crap gets into the codebase through updates. Especially when dealing with something hard, Claude will take shortcuts, that undermines the project's quality. In my projects, I deal with it by still pair programming with it. But as far as I understand, Anthropic deals with it by re-creating projects, rather than updating them.
They look at it mostly like a black box. If it works, it doesn't matter if the code underneath is janky.
Maybe the solution is just to hire someone who likes to fix janky crap full time and have that be their job. Honestly, I'd do it. I hate jank, so I like de-jankifying things. I don't think I'd mind such an arrangement personally.
I guess, they have some form of harness engineering optimized for Claude + they likely use more advanced models than those available to us.
And yes, they likely accept that not every line of code is clean.
But honestly (and unfortunately), average source code is nothing stellar either. I do SW dev for 30 years. And most projects I have seen were somewhat ugly.
> Every time I stop reviewing the code from Claude I later find janky crap
I'm very curious about that because that has not been my experience at all. I used to find janky crap from older models and I had to review every line of code very carefully. With newer models, and especially large ones like Fable, it just doesn't happen anymore. The code that I get is what I would have written. Actually, it's usually better because it covers more edge cases that I wouldn't have thought about.
At this point, I guide Claude Code, provide general direction, and make design decisions, but I rarely write code anymore. The code that I get from the LLM is just better.
I do not fully understand where the disconnect comes from. I do believe people that say that they get crap from LLMs, and I suspect I can guess some of the reasons for that, but I cannot be sure.
How can I put this politely ...
> I rarely write code anymore. The code that I get from the LLM is just better.
Coding chatbots excel at leaping low bars.
I won't even pick up on this kind of jab, and I certainly won't try to justify how and why the code is good.
> I do not fully understand where the disconnect comes from. I do believe people that say that they get crap from LLMs, and I suspect I can guess some of the reasons for that, but I cannot be sure.
Same here. There is an introspective comment below by khazhoux asking the question from the opposite experience. Why do developers end up on different sides of this coin?
LLM capability is obviously important variable. But those with bad experiences report using top tier models, so that isn't it. Likewise, level of experience can be ruled out. Each side of the coin have experienced developers reporting.
A lot of the other variables are not reported but I can go from my own experience.
Greenfield vs brownfield? The first is indeed usually easier but I get great results updating large code bases with decades of development. And I suspect developing greenfield with LLM is easier as the code is better written from the start.
Rarity of the subject matter? In principle, yet another XYZ app or web page is well represented in the training data so the LLM should have an easy go at making similar. In the past with low-end LLMs, this was noticeable. Working in relatively obscure programming languages produced just garbage. But, that was prior to the inflection point. And today I get good results working on relatively arcane scientific/mathematical algorithm as I do when I ask the LLM to barf out some web app.
Different levels of testing and validation and expectation? Even best LLMs make mistakes. The better LLMs make fewer and expose fewer still to the human as they create internal tests. Maybe people have different expectations and those with bad experience expect perfection from a single prompt? Perhaps, but any experienced developer knows bugs are a part of software development.
What else?
Whatever the answer, it's not a simple one and it may be a confluence of many smaller things. Maybe different developers have a multi-variant function in their brain that pops out a different quality-of-software score and also a different threshold that divides good from bad and all we see in their reports is that binary conclusion.
> Growing our monorepo
I think it is mostly this and my experience is that agents feel much more at home here. But on the other hand monorepos disadvantages become even more pronounced with agent work. It will become an unmaintainable mess, even if your agent can iterate through it regularly.
I still think building a pipeline that allows for separation of concerns is a requirement for maintainability. Sure, Claude excels to build software if I just give him a server where he can run wild. Any new requirement is an extension of the old repo.
But at some point you just have a giant mess and I am not really convinced of this approach, even if it certainly fits agents.
> how are they avoiding the janky crap? Or are they just accepting
They're obviously just accepting, have you seen any of their results? No internal insights required for these observations. Check out their clauded home page - try to open the right hamburger menu and click on 'policy' - it's impossible to do from the top because "Research" will expand on hover and push 'policy' down, and when you move further down, 'research' will collapse and 'policy' will jump up.
Likewise, the AI apps are infamous for their poor performance and buggy nature. Correctness is the price of such velocity
interesting observation!
on mobile it works (where I would expect the hamburger menu), and on usual desktop I have the usual menu bar, but when I change the window size on desktop I get the hamburger menu (and then the UX jankiness is evident, but for me easy to work around, as moving the mouse down closes the "research" block and then it's quite simple to move it up to be able to click "policy")
at this point I think Opus does something better by default
why wouldn't it work on mobile? As you hover your finger over the screen, the finger's shadow doesn't generate any events for the buggy website to react to?
By the way, the hamburger shouldn't even be there as there is enough space to fit the full menu (and even the company name) at the viewport sizes I've checked
as I was reading your comment I was thinking about what the bug is, and my initial guess (pulled out of thin ass-air) was that even on mobile somehow the touch targets overlap and there's no way to actually send the click to "policy"
Anthropic seems to have made progress on internal tooling which lets them review code better than average.
They get the latest models, they probably don't use the quants they give us, they have unreleased models, etc. We're not in the same situation. Maybe people who use the API idk
> This is my disconnect with this kind of speed, how are they avoiding the janky crap?
My understanding is that they have manual reviewers in phase 1 of their projects. Their goal is to refine the automation of reviewing, writing skills/rules, etc, enforcing architectural choices, etc.
Then in phase 2, the review phase is AI-driven itself and humans only get to be decision makers of escalated issues.
I've been building some game user interfaces with it, and on the surface it looks amazing. It's iterating at lightning speed, but then after a while you start noticing all the holes.
For example I've made a first screen with a lot of elements, and now I want to create a 2nd similar screen, so I tell it to use the same code as we've created for the first to make the 2nd. With that as input it will create it and a) use e.g. the variables and setup from the first b) copy everything into the 2nd c) copy some things d) make a generic parent for both to inherit from
A human would have realized that this is something we're going to be doing a lot, so the last is the correct answer. Make generics for all elements, e.g. the screen's panel, effects, buttons, text etc. then implement those along with possible overrides etc. for all.
But it seems that it has no clear methodology. Depending on your exact wording along with today's weather you will get any combination of the options.
> But it seems that it has no clear methodology.
As expected. That woukd require domain understanding, and that's way beyond the scope of stochatic parrotry.
> how are they avoiding the janky crap?
Not. The web UI is the jankiest web UI that I've ever seen. E.g. its mobile responsiveness frequently crops dialogs leaving the close box unreachable.
People need to actually put in some effort with LLMs here. Ask any SOTA agent to identify a specific type of "janky crap", and it'll quickly find every single occurence. Fixing it with the right pattern should be trivial for an experienced engineer, and therefore its agents.
Just do this for every type of "janky crap" and your codebase converges to perfection.
It's not that hard, but it takes a lot of tokens, and setup.
> Ask any SOTA agent to identify a specific type of "janky crap"
It's always total hallucinations. It could only see local minor issues.
It happily starts to introduce fake abstractions and other junior level "refactoring" making it even worse.
Seriously. I'm so tired of the "make no mistakes" trope. In reality it's "make X improvements" after the fact.
That's why they are doing so many commits. 3,000 commits a day, then 27,000 to undo all the senseless decisions Claude made.
They do say the following:
“Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.“
If my reviewers reviewed the “intent” of my code vs the actual code quality, I would’ve made Architect years ago…
Hm, that doesn't make sense to me.
The difference between intent and implementation seems like a very common source of bugs?
If intent and implementation was the same thing programmers as a profession would have been left behind 20 years ago.
Probably is a source of some bugs. But probably what they’re seeing is that it isn’t a major factor. Rather a bigger factor is agents trying to make changes were the intent isn’t aligned. And note, they don’t say they never look at the code. I suspect that valued human experience is having a feel for what types of changes is AI more likely to make mistakes with.
Many of the comments here seem to stem from the implicit assumption that before claude, software products were thoughtfully designed, secure, reliable, and beautifully engineered works of art.
If you spent some time reading Anthropic's docs on their website, you'll notice that sometimes the same feature has different intended behavior depending on which page you are reading and may or may not match the actual software behavior. You are better off using the docs as inspiration and figure out what the software does at runtime. And I have definitely done the thing where I just ask Claude to reverse engineer its own binary. I doubt anyone at Anthropic can explain what it's supposed to do.
So basically -- it's all crap.
To be fair, practically all modern large software codebases are like this.
I regularly run "experiments" on Azure to figure out what it actually does.
Well, Azure is really a work of art.
I'll save my nice words elsewhere.
I don't think too much of an insight is needed to understand that most of them 30000 code changes per day is just going around in circles, agent A introducing some kind of new abstraction, agent B finding and refactoring that abstraction, agent C figuring out that refactored abstraction introduced regressions, fixing and creating new tests for an abstraction, agent D finding out new abstraction was not needed in the first place and cleaning it up. Management s ecstatic seeing 4 merged PRs – time to make a blog post about merges per day growing more than tenfold.
Because they are using 100x more tokens with the latest best models only.
If you cranked up turns and only used fable level model for every interaction, you would get way way less jank. Plus when you have essentially unlimited tokens, you can always do extra AI review passes to reduce jank.
I think for us, outside of Anthropic, it is hard to imagine the scale of AI agents that they are running internally . We are talking about probably 10s of millions $ worth of spend monthly if not more.
lot of people here are criticising the set of features stuck together or "janky crap"...I don't know where you work, but I have seen a lot of that in all my career, and still do... and it was done by humans. The difference is that ai is doing it faster and can also quickly change it in the future, so not really sure it's the same level of problems. This is a big change in how we think about software development, bugs, and features tbh in terms of speed of reaction, identifying problems and caring about janky code or performance, because solving those problems, if they present themselves, is cheaper than before.
> how are they avoiding the janky crap?
I mean given all the sandbox escapes, and the leaked Claude codebase, I would assume they stopped reading their own code a long time ago.
> how are they avoiding the janky crap?
That’s the neat part! They don’t!
Honestly it does not matter provided u develop tests that show success. D u really care how its coded if it is amazingly fast efficient and solves the problem? Maybe in some projects but not all.
> This is my disconnect with this kind of speed, how are they avoiding the janky crap?
Because you are using LLMs to write code that you would be writing by hand.
Use it to write thing you wouldn't have time to write by hand and that are not mission critical: custom linters, support tools and so on. You know, each time you think to yourself "hey, I would write something to automate this chore, if only I could set aside one or two days".
You can also use it to document silly things you wouldn't take time to but that provide context to agents. Ask it to do a performance audit and to give you a few SQL queries and bash commands to gather stats about prod and logs. Then commit this document.
Will you read it? No. But your agent will use it to write code that's fit.
Keep doing this and 1/ you will be more productive, 2/ your agent will progressively improve so that you will let it do more and more.
This is the part that stands out for me.
> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
We still need the human creativity and spark in the loop to come up with these fresh, new, bold, creative ideas. I think if you let the current version of Claude run forever, it still wouldn't have the creative spark to create its own new source control system.
This is the future of a human software engineer's job.
Agree but every statement I now read from Anthropic just sounds like marketing bs to me. When they say 20x less and 4x faster they sound like Apple with no real data comparison or any detail at all
It's really incredible how they've squandered so much goodwill over the past few years.
While I think their models are great, and they deserve a lot of props for caring about safety (their interpretability work is great), their public tone just leaves so much to be desired.
Apparently they also don't understand the Streissand effect, given their recent blog posts about GLM 5.3.
> and they deserve a lot of props for caring about safety
At this point I believe they NEVER cared about safety. The CEO doomerism and the whole "slow down the development of AI" far cry is another marketing strategy based off fear
>We do 25 gazillion code checkouts per day and we merge 30 trillizillion code changes per day because we have 10 megazilion agents working on TEN MILLION projects at the same time in a 100 PETABYTE monorepo. Git is just not enough for us...
Typical anthropic post.
Every time any detail comes out of Anthropic or OpenAI it usually turns out whatever they’re doing is just incompetence. Right now I’d put money on that anything they were doing with git was incompetent garbage and they decided that the path of least resistance was to vibecode this thing.
You're talking about a company who sells software subscriptions of multiples of 1x, 5x, and 20x, without telling you the base amount of what "x" actually gets you, and changing it over time.
For the last year I keep reading from people and companies who say things like this, “I gave it one prompt, walked away for a while, and came back to something perfect and amazing.”
Meanwhile I’m here burning company tokens on top-tier models, and I still have to course-correct or problem-solve with my agents every 10 minutes…
English language needs a word to say “I don’t think you’re lying, but I also don’t believe you.”
The word is “bullshit.” Not as in “you’re purposefully making a fool of me” but as in “you’re not lying but your statements are uncorrelated with the truth.”
https://en.wikipedia.org/wiki/On_Bullshit
I've heard similar sentiment from a few people in my network. When I dig in, I find that these people are often working on much "simpler" problems. For example, they're building dev tooling (like testing frameworks) instead of working on business-facing services.
It's easy for them to one-shot solutions because the problems are very well defined, and the agents can easily reason about how the software will be used. The closer you get to the "real world," and the more complex the business logic gets, the worse the models get at making the right architectural or design decisions. If you're not watching closely to give input and steer the agent, it gets out of control pretty quick.
I find it ironic that these same individuals often loudly and publicly celebrate each new model release (as if they're early investors instead of early adopters), and tend to be very optimistic about agentic coding -- even though their jobs are the most at risk of being automated due to the simplicity of their work.
The closer you are to actual business outcomes and real-world software projects, the more you tend to understand that agentic coding patterns struggle to navigate the messiness of the real world. If you're working as a software engineer in an actual major industry (healthcare, insurance, defense, infrastructure, etc), you're probably going to be more skeptical when a gung-ho SDET makes bold claims about how LLMs are going to take everyone's job within the next 3-4 years.
> “I don’t think you’re lying, but I also don’t believe you.”
They're not lying. They're just a bit naive.
Today when you compare:
1. Fable as software designer and me as reviewer
2. Me as software designer
3. Me as software designer and Fable as reviewer
Then in my opinion, best design comes from 3, then 2, then 1.
Have you seen Opus 5.5 code? I don't know any developers who'd even remotely match it at a similar breadth. In fact I haven't found any discipline yet where it did _not_ develop an elegant solution, on top of achieving the goal. Senior human developers might be better in one or two specific niches maybe, but how many projects nowadays are confined like that
I do agree that Opus 5.5 (and Fable) beats me at coding, but I haven't found this to be the case yet in software design.
Where exactly do you see human creativity and spark in the loop or fresh, new, bold, creative ideas in
> We asked it to rewrite the whole thing, and with no further instructions
They simply asked the model to rewrite the code without suggesting anything to the model which counters your creativity argument, which I see many people make too, but which I believe is not correct anymore. Models IME already are creative in ways I personally find it difficult to compete with. I still occasionally find something that the model in the first iteration would not to but the model is already far superior with its capabilities than I am.
In the code to be rewritten. The stuff that happened before handoff to the machine.
Well, Anthropic used model A to output the version 1, and then used model B to rewrite the version 1 into version 2, so the handoff was machine to machine and not human to machine. I read that part of an article as if it was their internal assessment test of their new model B (Fable) capabilities so that they can see if the new model can find interesting things but by intentionally being vague and without implementing the bias into your prompt. I use this technique very often as well for two major reasons: (1) I don't know what I might be missing so I let the model do the unbiased exploration of problem space, (2) I am curious how far the given model has come with its capabilities. What Antrophic did in this case is an example of (1) and I wouldn't really call it creative since anyone could prompt the model with "rewrite this code".
But the first version was “mostly handwritten”?
The humans decided they wanted that specific conversion tool. The tool just worked on creating it. The decision is the human creative input.
That is not the quote parent comment was about wasn't it or am I missing something?
In case of your comment, I could have agreed to some degree but what you say is also not necessarily true since we don't actually know if the new source control is the result of human creative input or if it was also suggested by the model. Having worked with these models extensively I am inclined to believe that both can be true.
Well, the quote says that rhe first iteration was habd coded. So I assune that this means humans fully inchangé ofnthe décision-making; otherwise, the statement would be dishonest, wouldn't it?
Not to downplay Anthropic's engineering work here, but I suspect we'll see a lot more reinvention of the wheel, even at smaller startups, as AI makes writing code significantly cheaper.
The tradeoff between building something custom and adapting an existing solution is changing. Whether maintaining all that custom software turns out to be equally cheap is another question.
That many commits in a day I wouldn’t call it engineering. That’s basically brute forcing.
Its gradient descent all the way down.
I think the article would tend to agree with you, given its position on the reluctance of open sourcing and/or sharing the actual Scion software stack.
There's so much intricacy and subtlety in each organization and even individuals' environment being incorporated into tooling that expecting these tools to generalize at this point is perhaps an expired ethos.
I even see this in the tooling that I run in my own personal stack. Many projects are candidates for sharing, and I have nothing against it, but they incorporate such specificity toward my own environment, including things like hardware, that creating shareable and more generalizable projects from them adds more to the maintenance surface than I would care to support.
Not a reply to TT, but I just saw your nick and thought I'd respond. We both commented on a thread recently, and your response was really insightful and just a pleasure to read; I didn't take the opportunity to respond there (locked). You should, like, write a blog, man. I infrequently wish HN had a "follow" feature, but this one of those times. Peace.
ref: https://news.ycombinator.com/item?id=49768575
How about this:
https://hnrss.org/user?id=tcdent
What a nice thing to say! Peace!
When people wonder about where all the productivity of LLMs is going to I think this is part of the answer.
There is no doubt everyone is far more productive with LLMs in generating code. However, a lot of productivity is going into things like that. Have you always been bothered by the limitations of Grafana? You can generate another dashboarding solution that works best for you. You know that deployment team that is so slow and annoying? Now you can generate your own deployment engine. Have you always hated how kubectl lets you manage one k8s cluster at a time? Now you can ask AI to generate fleetctl. You can start your project with DynamoDB, rewrite it to mysql, then to postgres, then to duckdb. Start with Python, switch to Golang, then switch to Rust. Deploy on AWS, then move to Azure, then move to a hybrid option then throw colocation in the mix. Don't like your source control? create a new one. Json is too limiting? Invent a hybrid JSON/HCL language. The browser sandbox is limited? Make a desktop app, make a language, make an OS, make a kernel.
> However, a lot of productivity is going into things like that. Have you always been bothered by the limitations of Grafana? You can generate another dashboarding solution that works best for you. [...] The browser sandbox is limited? Make a desktop app, make a language, make an OS, make a kernel.
This is one of the core reason that I dislike the LLM hyping. People are just reinventing stuff and when you ask them, it's actually a very subjective reason. Ok, anyone can do what they want with their time, the issue is that they want people to follow in their steps.
Not saying that you need a novel idea to be worthy, but those random projects have no guarantees behind them other than some cursory testing. All software have bugs, but I can't interact with something that is randomly built.
I don't see the problem.
I am one of those people that reinvents the wheels because external ones don't fit my use cases.
The fact I can now fork any software and do these changes I always wanted but had no time to is amazing to me.
I do not pretend to sell it or share it, as it is de facto void of any value whatsoever.
> I am one of those people that reinvents the wheels because external ones don't fit my use cases
That’s OK, but I’ve yet to see someone articulating what exactly doesn’t work in the old product. It’s always “Let’s rewrite or reinvent 90% of it, then add a minuscule variation”. The variation may be interesting, but the reinvent is what’s make it untouchable as no one (including the author) has time to ensure things are right.
A patch or a plugin is ok, but not a whole new project with dubious claims.
Why do you care, it's not like anybody's shoving it down your computer.
> Why do you care, it's not like anybody's shoving it down your computer
They are trying to get in every opensource projects, increasing the burden of maintainers. And once a ban is announced, they’re here protesting.
I agree. I think if everyone doing that is doing it on their computer like you're then it wouldn't be a problem.
When other people start doing that to other teams at work, other people in open source, other people in professional settings, etc it becomes a problem for various reasons. The idea that I'm gonna gift you with a 75k lines of python to "just run it" and ask your agent to hack on it might be fine (who cares as you put it). Now scale that to 47 of such projects (a month)
I think the point argued is that the hype tells us AI is worth a shit ton of money (see anthropic/openai/etc valuations) but that a lot of the benefit is in economically marginal use i.e. make some dev's life a little bit comfier.
"Someone is wrong on the stock market." Isn't there an xkcd about that?
386
it's wasting limited earth resources
Eh, I could be quite specific and detailed about the "things that don't work" that are motivating me. Because most of my prompts require actively thinking about at least one of them.
> what exactly doesn’t work in the old product
Assumptions. I've rebuilt a number of inference engines to run on windows and unify the dependency venv to last version to save space, unrolling model blocks so they fit in my 16gb card etc. A lot of thing thay I was locked out are possible now as i am no longer held back by invariants decided by someone else, and because I do not have to spend time to get things done. I've changed games to have a less grindy loop, added features, or making learning them easier. Ive fixed drivers, patched versions, brought dlss5 to game weren't supposed to have it
It's not ground breaking, but its liberating.
you know that you can reinvent the language that youre talking. but the value of common language is that you can use it to interact with other people. same with software its the synergy of common understanding.
A major benefit of LLMs is in expanding scope of things you can do without involving other people.
Communication and coordination are bottlenecks.
You wont be able to do everything alone, and with llm you can push the barrier so far that eventually when you will need to communicate, no one will understand what youre talking about :)
There will be no common framework to build on. Like imagine talking now to someone from ancient rome, they will not have an understanding for what most of everyday words you are using today.
Fortunately, we already invented the universal translator :).
its not about translation. its about the concepts. there was no concept or microbe, elcticity, gravity integral, rational number etc. so imagine everyone now would have totally different concepts that only they understand. how do you communicate without common ground.
You build one.
For some things, you'll need to build a working framework from grounds up. Electricity or more complex maths would be an example. For some, it might be enough to repurpose the existing framework they already have. For example, microbes: if you're trying to communicate something about health and hygiene, chances are, the other side already has a half-decent framework built around malevolent spirits or taint, that amounts to the same thing in practice.
Or, just tell the universal translator to ELI5 it to the other person :). Or "explain like they're a caveman". Our UTs are fully capable of translating concepts between conceptual frameworks.
There is surely a tremendous amount of waste and noise right now. But does it _actually_ matter what the code looks like for a desktop app that’s useful to a small handful of people? Every single implementation has at least one person who thinks it’s dog shit, no matter how correct or perfect it is. We have to get past this. If you really can’t stand it, build the thing yourself or some deterministic rails to reel it in.
And, there are also people who know what they’re doing building things they couldn’t before.
This situation is a net positive, and not too different than before. It’s up to you to navigate the noise.
> does it _actually_ matter what the code looks like for a desktop app that’s useful to a small handful of people
Not really. But equally, I imagine most people have a similar experience of software development to me - complex, messy and confusing code also often has poor performance and a high rate of bugs.
I have done several of those things on the list in just the last three weeks. It's amazing. Never been more excited about building.
I’m sure it’s exciting. I’m wondering if there is any value.
Sure there is, you said it yourself. You can make all those changes because it's easy, the lock-in effect is waning. We can move around and take better deals and choices.
This is actually a scary side of AI for providers: we use our own AI to research better choices and escape dark UI patterns and vendor lock-in. That information asymmetry and the difficulty of leaving a provider used to be their moat.
On a system level I think AI code is more like biological diversity, it hedges against monoculture risks. We don't all fall down at the same time if we don't replicate the same code.
I don’t really see what lock-in means for 90% of what AI is replacing. It’s more of a fear of commitment which isn’t exactly new. Enterprise space architects have designing systems to have pluggable everything from language, frameworks to carbon based life form.
As for the bio-diversity, I guess we’ll see the next time us-east-1 goes down.
It's surely subjective and changes project to project. I built a personal app on iOS with GPT last month because I didn't really like any of the flash card apps I tried for free. The way it works works very well for me and if I have any feature requests, I get to actually tell a thing to implement them, rather than sending a dev an email and hoping maybe they'll consider my idea.
At work I've been building an end to end system to do a certain process. The codebase is definitely more spaghetti than ideal and I really am concerned about maintenance down the road. But on the other hand I was able to POC it faster than anyone on the team would've before LLMs came along. We're already getting useful results from it too.
There's value to me! I've never been happier about my tools.
Your post above describes the value. All the annoying restrictions and UI/UX decisions made by SaaS providers that don't work for you. The value is being able to essentially wish into existence your ideal tooling, unburdened by the tradeoffs and decisions that some product designer thought was best for the masses
> decisions made by SaaS providers that don't work for you
Unfortunately this starts to break when there's more than one you on the team.
I see a lot of hand-rolled NIH solutions that create value for one person only, and are a net negative on the team.
In a few days, I created a few apps with AI. One saves multiple mechanics 5-15 minutes per day and another can save multiple hours of mistakes during the project, and one saves a lead engineer 15 minutes per day. These savings should add up over multiple years assuming all keeps running as it does now. I’m having good hopes because there have been few bugs so far. Mostly just tweaking.
Enjoy your maintenance responsibilities and hopefully compensation.
I’d never tell people what to do with money or free time. If someone wants to build many apps to help them personally, all power to them.
I can’t imagine a worse hell in a team or collaborative setting though. Imagine (or you don’t have to imagine since it’s already happening) asking a coworker “how they check how feature X is doing in prod” and their answer isn’t a query (regardless of how long it is) or some slightly complicated script to run. Instead their answer is a 27k line custom Rust desktop app. Then 2 days later you discover that the big green “All Systems Operational” is just a bit of static text, and in fact not all systems are operational. The app just doesn’t check things correctly.
Originally I felt that there was going to be a reinvention of the wheel, until I saw the "shape" of software development changing, and the tooling needing to support the new flexibilities in positions and process.
Software development is not as much de-risking at the front like it used to be before building.
Now teams can build lots of stuff that works great, except it's the wrong things.
Most things are becoming the understand the problem "shape".
Understanding the shape of the problem isn't something LLMs can always make as.. svelte as early on.
Word generators gotta generate..
> Now teams can build lots of stuff that works great, except it's the wrong things.
That used to be called "time wasting". If you want to build software that will be used, you need to interview the users. Maybe a few things does require research to nail things right. But most software are user-facing, the research side is about optimization.
> Most things are becoming the understand the problem "shape".
Then should we be so reluctant about "trialing" invented solutions. It's the same thing with most AI projects, they're trying to justify the project after the facts while being very reluctant to get user insights.
Time wasting is a definite concern - especially when vibe coding doesn't see the impeding brick wall(s) it's about to hit.
Vibe coding can be kind of a new type of cowboy coding. It could can be harnessed productively if done mindfully.
Interviewing users is critical, more importantly solving problems worth that people are lining up to pay to have solved before writing a line of code is critical.
Trialing solutions is always worth learning from, I usually ask about the problem that lead to building it. Sometimes that can be too narrow or vague of a story and you can learn how much of it is a solution looking for a problem, or vice versa.
We're trending towards just building the thing fast and getting feedback sooner on it. It's interesting to see end users actually trial a UI very early on instead of just being shown designs.
In addition to that, solving for every approach to a problem to see what wins is also very interesting.
The key, still, is shipping far less, too soon, to minimize cruft.
> If you want to build software that will be used, you need to interview the users.
Some people write (I hate the word "build" here) software for non-business purposes. As nice as it would be to interview potential users, there's no clear way to discover and contact them.
In the educational universe, this "interviewing" is more common than not. Teachers are asked to give feedback on textbooks, calculators, and software, including lots of educational half-baked A. I. garbage. Some even offer us food for our feedback. Can you give an example of a type of software which is developed with "no clear way" to contact users?
> Understanding the shape of the problem isn't something LLMs can always make as.. svelte as early on.
My suspicion is that this simply won't go away as the models get better, because it isn't a question of how good the models are. It's a question of the human operators knowing exactly what they want. Which is, I conjecture, usually impossible until you try something and critique it properly.
This was always a big problem even in the pre-LLM days. The difference was that you only needed to be better than your competitors at solving the customers' problems. There's a reason everyone gripes about the software that they use and it's because the software is built to satisfy a very broad customer base, broad enough to justify the engineering time put into it. Now you're competing against tens (maybe hundreds) of other firms all building for the same competitors so there's a lot of pressure to get the shape exactly right.
There's a reason the product-minded highly-technical engineer was always highly sought out.
Recently a colleague of mine shown us a demo using a fresh model designing a building construction detail live, in minutes, which is a function that we struggled to bring into our construction design software due to resource limitations (and legacy barriers) for long. All this in the context of offsite constructions where the design can directly be exported to the right CNC machines in the factory, and arranged for delivery and assembly order. We have let ourself amazed, despite the controlled demo situation and the small detail, it was a remarkable performance. For a minute or so, but thinking further, why would the customers (construction design engineers) of our competitors flow to our legacy software full of legacy pitfalls they know about and have to live with when they can turn to AI for a fresh design solution themselves, just like we did? They know the context better than us anyway, just do not have time for bit-carving. Why would we have customers when customers of our customers can turn to an AI for a design that is buildable? I was consulting one model about a structural engineering detail, and it was quite well informed I have to say. From training data alone. I felt, maybe I need to rethink the rest of my career, that is not that long but still uncomfortably long for being predictable beyond a few years.
Well, things are not that bleak, probably, at least not for our customers. Regulations (accountability) and physical world matters still require them. But not us for our customers, not that much and not for too long. They tend to be the tech savvy folks eager to jump at adjacent domains when it saves them a percentage or two off the construction costs (not a small pile of money), and AI with its programming language of English and deep knowledge pool is not that big a jump from their standpoint. Our domain is pretty well formulated, regulated, documented, inherently built on do-check-adjust loops already. With a thing like AI it might worth to try making and improving a design solution in a second loop. But they also started to invest in robots already driven by software for their factories, AI might jump in there, and I wonder how much more until the regulations are forced to change by Super Intelligent (R) politicians for the sake of AI accelerated built environments (housing crisis, you know!), shifting the AI use further out towards generic population, wanting to order a new two bathroom condo for next Friday from Alibaba. The conservative construction business is a nuisance and obstacle in the way of modernisations anyway, go around it!
I will go to work tomorrow, and for some time, but the picture above feel less and less fictious with each passing day of AI news and usage.
The question was never whether you could do this or not. It was always whether it was wise to do so or not. More than one company has gone under writing their own editor, compiler, database engine, operating system, source code control system, ticketing system, CMS or ERP.
Congratulations you just burned a ton of tokens to get right back where you started!
Building something custom often is re-inventing the square wheel. I'm dreading the idea of switching jobs only to find the LLM generated software equivalent of Terry Pratchett's “Ordinary madness he could deal with ... But Vorbis had passed right through that red barrier and had built some kind of logical structure on the other side. Rational thoughts made out of insane components.” from Small Gods.
This was already a problem in the industry, many companies reinventing the wheel on larger or smaller scales; with AI it's becoming easier to do so. Ultimately it's fine as anything not successful will just die out again.
But also, some of these internally developed tools get published and end up pretty successful.
Previously there were two problems with NIH solutions:
- they have implementation cost.
- they are usually born from arrogance mixed with lack of understanding of existing solutions and their tradeoffs, and typically end up being a net negative.
Before LLMs, the cost is what primarily used to stop us from building them, it used to be a natural filter for bad ideas. Now that the cost is gone, it's nearly impossible for most teams to stay disciplined and not be building those.
Why wouldn't you build a software factory? If you succeed, the ROI is infinite. Who cares about immediate customer needs, we're building a utopia here. Once it's done, all customer needs will be solved. Just one more month, I promise.
"Not Invented Here" syndrome will be back with a vengeance!
> In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day.
> The history of our main branch alone is almost 100 GB
What value do you get from carrying around all of that history? Your code is changing so rapidly that it looks absolutely nothing like it did a year ago, so at what point are you going to want to see what it looked like back then and every single change that happened to it since? Wouldn't you benefit more from creating a tool that can semantically truncate history entries that are no longer relevant, since even with scn your history will continue to grow out of control?
I think your lawyers might have a word with you, too. Source code history is more than just a record for developers. It's also a historic record that can become relevant in legal disputes.
Relevant, yes. Advantageous, maybe.
If you're Anthropic, you probably don't want a 100Gb of haystack lying around in which a damning needle might be found.
While true, if the 100GB of history that isn't important anymore is dragging you down, create a new repo with today's code and move from there, then archive the old history for e.g. legal reasons.
Legal argument is already a strong one, but I feel that the training data in there must be invaluable. Such a mass of shipped features, regressions, refactors, complete rewrites and pivots, all with full internal history, is a goldmine. Whether archiving and switching branch might be in order, that's a fair discussion though.
Every now and then I like to browse the claude code release notes just to get a sense of the number of changes in that product over time. There's little choice but to simply count bullet points if you want to try to quantify it, but it's interesting how that count over time compares with the charts in this article:
https://imgur.com/a/M30pFLc
It would be interesting to segment that graph by bug fixes vs changes, and maybe also significant changes vs minor/trivial changes.
i like to cross-compare and see how many items are marked as Fixed or Added.
https://i.imgur.com/NEhgPTa.png
Nice!
I think one thing is consistent across all of these visualizations: they moved the needle. Hard to say what that needle points at or where the redline is, but fleet change is afoot.
More fixes could of course be downstream from more bugs
> Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend
I appreciate that they're extremely self-aware here.
> would realistically want to spend
...on expensive / over-priced Anthropic products.
If you wanted to use this amount of tokens you would find a cheaper provider.
Still some room to be more self-aware.
> extremely
They briefly articulated one obvious, undisputable and toothless truth about themselves. This is far, far from "extremely"
> Very often, we found that the role of the engineers was to tell Claude “no.”
That line stuck with me. I wonder how the engineers building Scion feel about that? If that really is the future, then I guess I have to find another profession. Retirement is still too far away.
I feel like my role is rather to say "yes". Claude wants to keep all that legacy crap around and I keep saying shut up and purge.
I have found this to be true as well with modern claude, and to some extent codex. The blog summarized it perfectly — perhaps as a result of so much RL on security/adversarial review, the models currently will speculate a (valid, but unlikely/out of our design space/deep edge case) problem, and then cheerfully triple the codebase size and cut speed by 10x in order to fix it.
It feels exactly like talking to an experienced security engineer suddenly tasked with designing a large system — right now the tradeoffs that an architect makes don’t tend to be there, so quality oversight gives much better outcomes.
This article doesn't do a very good job of convincing me that git is technically incapable of handling the described workload. There are a lot of steps between vanilla git and building your own VCS from zero.
If I was an investor I would be on the phone with Dario asking why we are wasting engineering talent on reinventing source control and churning the product with 30k pulls per day.
>If I was an investor I would be on the phone with Dario asking why we are wasting engineering talent on reinventing source control and churning the product with 30k pulls per day.
To be able to write PR pieces, like TFA.
Indeed, while struggling with VCS is a right of passage for any tech company, I'm not sure why what they described couldn't have been built as a diff/patch API in front of git.
How else can they show their superior model?!
Surely they can't manage to have a performant binary versionnof claude code and they're stuck with JS runtime
This is why you are not an investor. (Cue McMahon meme), A marketing campaign to get eye balls across all the major tech outlets and take up headspace in everyone's trending feed? Oh yes. A potential, closed source alternative to github you can slowly wrangle your users to move to? Hell yeah. An always online centralized design requiring users to stay connected with the server for every operation? Let's go!
They had a CI/CD problem, so they fixed... version control.
They've ripped out and replaced the most carefully thought-out part of the CI/CD chain with something less carefully thought-out, while leaving the teetering house of cards in place.
Yes. In a world of bicycles[0] and horse carts, they realized neither are sufficient for their needs of cheaply and quickly hauling heavy loads across long distances. Instead of leveraging "the most carefully thought-out part" of using bicycles, they slapped a steam engine onto a cart and fastened it with some bicycle part, and guess what, it works, and hauls more stuff faster and cheaper than either bicycle or horse carts ever could.
--
[0] - If you can even call anything built before modern petrochemical industry a "bicycle".
More like in a world of round things that can bear loads, they decided to... (checks notes) reinvent the wheel?
Their whole point is that none of the existing wheels can actually handle the load they need.
I don't know why is this even controversial. Agentic swarms break scaling assumptions all across the typical software development life cycle; VC and CI/CD are just few obvious cases.
They (allegedly) had a problem or copying their mono-repo around their network so they solved it by making the server do all the work.
Sounds like a out of the frying pan and into the fire kind of solution.
With the added bonus that you now have a vibe-coded VCS, so good luck when that has strange bugs.
7 engineers is hardly a massive resource allocation. Even if they produce nothing of value the lessons learned along the way will be valuable - They're essentially dogfooding agentic swarm development which will help inform their future product direction
I am about 80% certain that using well configured Perforce could work just as well while saving Anthropic some money, considering the salary of Anthropic engineers.
But then Anthropic doesn't get the PR, so...
Anthropic is at its core an R&D company; they invent things and sell them. If they invented a new version contol system they can sell, or if they solved a bottleneck that prevented the ability to develop stuff faster and better, it's a worthwhile investment.
The cost of innovation or "just" writing a new tool is plummeting fast.
> The cost of innovation or "just" writing a new tool is plummeting fast.
Doesn’t this make it less profitable to be an R&D company, since there is less market share available?
What is are the benefits of supporting extremely high churn rate in version control? What is the code doing? What problems are being solved? Are changing requirements the driving factor, and if so, why are requirement changes accelerating?
> ..., the latent demand for merging code is practically infinite, and every bottleneck we remove immediately leads to a further increase in the amount of code we produce.
If the goal is to write software which solves real problems, how can the demand for code grow infinitely?
Surely their goal is ever-faster recursive self-improvement.
It looks like they are recursively gradient descending into their own local minima at increasing code velocity.
That by itself doesn't imply need for infinite code. It may just imply the opposite.
Fortunately they realize that code can be deleted in a commit, too.
I find the accelerationist mindset utterly ridiculous. All that busywork and for what exactly? What effect has all this overproduction of LOC had on the physical world and on our existence (beyond the externality of erecting huge data centers in every corner of the globe and the consequent ecological impact)? Is software getting better (in terms of quality, security, efficiency)? Are we getting better government services? Are we getting better food? A better environment (silly question)? Are we working less, as tech titans keep promising us? Are we getting anything better? Not to mention all the negative aspects of this technology and the financial apparatus driving it.
I agree with the sentiment, and the answer to all these questions is usually ‘just wait 6 more months’. Like what is tangibly, actually better in the real world right now with AI? Whenever I ask this question people pipe up with a side project or a new SaaS, but that stuff is just not very important compared to the things you called out. Food, services, QOL.
It seems to me like the AI labs are just pushing the envelope with blind faith that”something” will happen if they just get enough development speed. Nobody seems to have a coherent idea of what that something is.
It took a long time for the internet to result in things that improved life for in tangible ways.
Damn near a decade or more. We're a little under 4 years into the chatbot era.
But the old internet never had so much negative impact on the world with so little upside.
Facebook and its ilk sort were more or less the unravelling of western liberalism. The descent happened before generative ai.
That does not make the AI-part any better.
Seems to me the goal is to create a convoluted spaghetti mess in every code base so that humans no longer have a chance to understand how it works and thereby force all software development to go through LLM obfuscator-deobfuscator cycles.
> Like what is tangibly, actually better in the real world right now with AI?
I took a photo of a tree with chatgpt recently and asked it what species it was. That was kinda neat I guess. Someone forked a ps4 emulator with a bunch of Bloodborne specific improvements so its more playable on my setup. It's kind of okay for medical stuff.
It is really laughable when you zoom out and realize that nothing much has really changed day to day
> I find the accelerationist mindset utterly ridiculous. All that busywork and for what exactly? What effect has all this overproduction of LOC
Yeah I wish this whole energy was towards optimization and improving hardware.
One thing that people are finding hard to believe is that the frontier labs are cultists who want to "create god". Hence the focus on improving intelligence so they can get to recursive AI, as well as the fear-mongering.
Aanthropic-openAI duo cannot go bankrupt soon enough. This industry is going to flourish with those two out of the picture. They're the pets.com of 1999 trying to have a say how the internet should be.
> Yeah I wish this whole energy was towards optimization and improving hardware.
Have you considered they are doing that, and also all the other things, too? That's where all these tokens go.
But I guess it's easier to believe they're all cultists who like to watch GPUs spin for no reason.
> All that busy work and for what exactly?
All that busy work was for going from zero revenue to $50B in revenue in about 3 years, and releasing AI systems that are getting better and better at a ridiculous rate. AI isn't solving all the world's problems but you can't deny that they are extremely successful with this method of operating.
It's a fair question to ask. If literally all the economic growth (and then some?) is going to AI companies, then nothing else is improving, in the aggregate.
The broader question is definitely relevant, I was thinking more in context of the original post justifying their need to re-invent source control.
I definitely think that's an interesting question though. I know that in the physical product development world I'm seeing really odd asymmetric speed ups. Creating simulations is essentially automated at this point, stuff that would take days to code by hand can be done in an hour. Firmware debugging feels 3x faster than before. But the big slow stuff like manufacturing, assembly, and testing are completely unchanged. All of that little stuff probably adds up to maybe a 5% faster product development cycle than before if you're really leaning into AI. It's the type of thing that will compound over time especially if it's happening all over the economy, but won't blow your socks off.
I think part of the concern is around “could it be better”? Like if an elite athlete eats junk food all the time. Sure it’s working, but is it because of the candy? Anthropic is often criticized for their poor UI/UX. The models are undeniably powerful but the packaging is typically poor. This is not to drift into the conversation of profitability either, to be clear.
But the revenue is just because they’re selling at a huge loss and are being allowed to do so because lots of stupid investors are betting that one day they won’t be selling at a loss. That’s not economic growth. Anyone can have revenue selling diamonds for a penny
I think their success is pretty debatable. This is a company that lost 8 billion dollars last year, is in an extremely expensive race with other companies, and is basically the premium version of something that's now becoming a commodity. They might be doing some impressive work but as a company their outlook seems questionable.
Since when have we expected companies scaling that fast to make a profit? The fact that they've had profitable quarters already is pretty insane.
There are definitely risks to their business long term from commoditization and a lack of a moat, but I don't think that detracts from my original point that they are building something highly desirable with this new version control system today. They are probably the fastest growing company of all time in terms of revenue and valuation? They're not just slopping out more and more LOC for no reason, which was the original point being discussed in this thread.
To play the devils advocate here (just for software), I would say principally this:
"A world where starting is incredibly easy and finishing is more complicated than ever"
LLM's do allow you to oneshot huge starting bits of work in record short time. The frontier of the amount of work they can oneshot is clearly getting larger and larger. But I am still not convinced that they allow you to maintain existing work at a cost that is below a human being.
But, if the frontier of what you can oneshot continues to grow unbounded, then it really will not matter. We will eventually oneshot all software and maintenance will be a thing of the past.
... But that is a very big bet.
The funny part is, no one is making any money off this either. Traditionally we’d say that profitability is not necessarily an indicator of real value for humanity, but with LLMs we don’t even have profitability! The only people making more money here are manufacturers like Nvidia and data centre companies
[flagged]
What defines a "real problem" but that someone is willing to pay money for it?
Who decides what the "real problems" are?
Imo most of the economy and wealth creation is around activity that is not solving a "real problem", but who am I to say if someone else is willing to pay for it? Presumably, for said person, the problem is "real" enough.
> who am I to say if someone else is willing to pay for it?
You are a moral actor capable of rational thought. The very image of God themself, if you like. Don't shrug off your own agency.
Yikes. So much evil done in the name of God. I would think nothing would be further from the conduct of a "moral actor" than equating themselves with God.
Completely unnecessary. Secular humanism for me, thanks.
Christ is exactly what we need right now (as always)
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."
"Don't be snarky."
https://news.ycombinator.com/newsguidelines.html
There's plenty of room for thoughtful criticism and skepticism in HN threads, including the present one, but please don't post like this.
This is the main thing that raises my eyebrow.
If you are constantly churning your codebase to the point that you need to invent a system that handles your churn... then you must at least be admitting that the previous 6 refactors you wrote were not good, but NOW you have it right.
I feel that good, well designed code change by accretion, not by rewrites. Code that remains stable for a long period of time was likely better than code that needs be continuously rewritten (at least with humans writers).
But maybe this is just a human-centric view of what good code means? That LLM "slop it until it works" approach to development will have a different measure of what quality looks like?
idk, over the past 2 weekends i have a repo for hacking on fujifilm camera firmware that's gone from zero to tens of gigabytes: i've got an emulator for older arm cpus like the one in the X70 (2016), a Hypervisor.framework VM for running their newest AArch64 software for X100VI (2024), a complete C re-implementation of the x70 source, many tools for injecting code and executing on live camera RAM over USB. the emulator has gone through 3 rewrites: first it was python and naive, then it was python with hot parts in rust with a more mature device interface, and now it's rust with some cold or experimental devices still python.
i really don't want to brick my camera, so the deterministic emulator stack lets me do integration tests end-to-end. if i was doing this by hand with no ai, i'd probably have to run stuff on the camera just to test it. but because ai, i get deterministic simulation testing, at the cost of needing to burn $100 of subscription on it.
ai code is more like clay you gotta continuously keep shaping over time, but you can also get very very far, very quickly if you decide the code itself doesn't matter and all you care about is results.
Agreed, a buddy and I wrote a simple mod that shimed into Warhammer 40K: Rogue Trader and were able to pull all of the art and description assets for that game by walking the object graph. Took about 4 hours total (including the site).
Similar to your camera, we would never have been arsed with doing that much work were it not for LLM vibbing. It allowed us to get something working (sufficiently) very quickly.
At what point of maintaining this code would it be cheaper for me to actually find underlying patterns and program this myself with some actual thought, rather than continuously "shaping the clay" of my LLM slop? These questions keep me awake at night (not really), and probably predict the future of software engineering!
I'm curious, was there a specific reason you went with that solution for grabbing the games assets?
Because asset-ripping is a pretty widespread, well documented practice in gaming - it's how a lot of modding communities figure out and expand on the games they mod.
A quick search shows WH40K: Rogue Trader is built on Unity too, so any number of existing Unity-specific asset-ripping tools should have worked for your purposes. Though I should say I've never actually dabbled much with modding Rogue Trader, so maybe I'm wrong?
It was January I think and I don't remember our thought process. I think we mostly just remember looking at the fandom.com or whatever crap was there and thinking how out of date that stuff was.
We also just wanted to try some modestly sized project with an LLM. Neither of us had any real experience doing any of the things we ended up doing for that project (other than my making the website and him understanding the graphics pipeline).
So, unfortunately, no, there was no specific reason I suspect.
https://stephencagle.dev/posts-output/2026-01-30-warhammer-4... is my best recollection of the thought process.
Are you planning to release any of this? Fuji cameras are so nice but need some opening up
hopefully! i’ve got lots of good stuff for the EXR2 processor generation.
> then you must at least be admitting that the previous 6 refactors you wrote were not good, but NOW you have it right.
What does that remind me of?
Sounds more like your thinking is that version control is only for production code.
I think that they're essentially solving the wrong problem.
Imagine that you have git but you don't even have a concept of programming language more complex than assembly. All your commits and git history are pure assembly. Everything is cool, until one day C++20 randomly spawns into existence, and a bunch of guys decide to use it in the company. The issue is, the compiler generates so much assembly, that your existing workflow cannot keep up with it. Your assembly git repo grows into hundreds of gigabytes, with PRs that are pretty much complete rewrites coming every single day and there's no way to review any of that shit.
What Anthropic is doing is redoing git from scratch so that it can handle all that assembly, while the correct thing to do is to commit C++ code into the repo, with deep trust that you'll never hit a bug in the compiler - basically impossible for initial versions of compilers, taken for granted nowadays.
What I'm saying is, Anthropic should create a versioning system for prompts, because they are the true "source code". Given the same prompt and the same sequence of random numbers, LLM will generate exactly the same code, which sounds awfully close to how compilers work. Once in a while put the generated code as a point of reference+ caching and presto.
Except they don’t right? They aren’t deterministic. All the hypemaxxers over at twitter keep saying you need a way for the agent to verify the result itself. Would that be another prompt then?
No one is actually one-shotting new code on top of the existing repo and just checking it in. Even in a simplistic case, it’s multiturn. You give it a prompt, and it writes some code, tries to build it, and it doesn’t work. So then it keeps iterating, and changes some more code until it builds and then does the same thing again until it passes tests. And if you have a complex system, it might do a lot more CI.
The input to this is not just the preexisting code, the prompt you provide, and some random seeds. It’s also the output from every single tool invoked, which means you need to record the state of every single tool and every file that was read, every API that was called. It means that you can’t upgrade Claude. It means that you can’t upgrade the compiler. If you change anything in the state of the world you no longer have replicability.
Aside from all that, I don’t know why you would want this anyway. Storing a bunch of prompts doesn’t seem to actually gain anything even if it worked.
> What I'm saying is, Anthropic should create a versioning system for prompts, because they are the true "source code". Given the same prompt and the same sequence of random numbers, LLM will generate exactly the same code, which sounds awfully close to how compilers work. Once in a while put the generated code as a point of reference+ caching and presto.
Bad analogy, cause this is not how tooling works. Tools beget more tools - there's value in specifying "these are the sets of tools we need to make", but unlocking new capabilities doesn't happen unless you... make the tools. And use them. And keep them stable. And watch them break, or become obsolete, and build on top of that. The world has state and path dependent.
> Anthropic should create a versioning system for prompts
If LLMs were deterministic, like compilers, you would have a point.
Erm, don't ask questions, you luddite. The real thought leaders tokenmaxx as they ride the vibes.
Finding solutions by applying Brute force, thus the need of extreme token usage and zilloons of CI/CD tasks per day to verify the deliverable.
My thought too. Like, I get that Claude gets new features pretty frequently but.. what on earth are they possibly doing that needs that much churn?
The only thing I can think is that they're doing a google style mono-repo, which like, fine, that's a thing to do, but I really have trouble picturing that changing code at that rate is a useful thing to do.
unshaved jak walks into the room
The churn is rewriting 50+ line comments on every function. These comments will also include every place that references this function, as well as random historical data.
Simple, they want Recursive Self Improvement and worry that git will be the bottleneck. So they are making source control as efficient as possible in preparation for that day.
From the first paragraph: "Claude is able to autonomously iterate on research loops, micro-optimize complex programs, and build new services from the ground up at a speed and scale we haven’t seen before. The bottleneck, now, is all of the developer infrastructure built for human scale: source control, builds, and CI."
As I can see it is natural way that big tech will concentrate their source control into powerful server side, Facebook did that (their monorepo use a customized mercury), and now Anthropic. But I believe we should remember what git has changed is not the way we commit code, but more the way we share code and open our code. Before git, there is cvs/svn and it has never given us the real distribution system as Linus ever talked. After git, besides the speed, the most important thing as I believe is that any one can own the full history of a repo, and that's the foundation to true open source. Anthropic now? a backward step to a closed garden that's all, and they have all the tokens, so let them be happy with themselves, that's all.
I came into this blog post a hater, but honestly - hats off to the team. This sounds like it was a very arduous project, and I don't envy the engineers that have to sweat over handling Anthropic's scaling commit volume.
To that end though, this line feels like it (intentionally?) undersells the engineering team -
> Very often, we found that the role of the engineers was to tell Claude “no.”
To me, this blog post highlighted the immense value that domain-specific knowledge still carries in engineering. I wouldn't be surprised if their team was comprised of former Facebook/Google/Github source control engineers. Individuals with real experience on these types of systems.
The dream for a lot of folks in the industry right now is that you could just staff a bunch of generalists on a problem like this, and through the power of Claude come up with a sufficient solution. I think the reality is that without the domain expertise to direct Claude, you would end up with something vastly different, and not really usable at Anthropic's scale.
> as of September, we merged over 30,000 per day.
There's going to be an ugly phase where everybody FOMOs into thinking they need to do the same, only to realize that they're not Anthropic/Google/etc.
I genuinely don't understand the core value they're pitching here. The tech is interesting, and I'm not trying to under cut that, but I question the idea that this is directionally valuable advice to nearly any company.
This amount of churn feels like a thing only possible with no real profit motives and a lot of money to waste.
Lots of activity is not necessarily synonymous with productivity, and it carries real cost.
So I think the design here is interesting, and the concepts seem applicable to where some agentic development may end up, but I question the underlying principles that drive this being the right solution space at all. At least if the goal is a profitable company.
I think its pretty straightforward no? If agents are doing all of the work, they'd probably need to not have bottlenecks. This is going to be any company in the future, not just Anthropic. Wouldn't you agree.?
any company, merge 30,000 code changes per day? do not agree
30000 a day seems already beyond absurd, it's one merge every 2.9s. I'm sure they will soon have more than one merge per second. From the graph, the rate was slightly lower but already insane as fuck the whole summer. An just regularly high to insane high sooner this year.
Yet IMO the admin controls Claude provides just started to be adequate for a small company a few weeks ago (and even so are still ridiculously fragmented and seem fragily designed and not exhaustively documented), so I can't even imagine for a big company.
Do you need that rate of change to make basic controls? I'm not sure. Do we see such dramatic fast evolution in the products? Not really. Are bugs we report actually fixed? I tried once and it just auto closed.
Anthropic should optimise for something more interesting than number of merges per second.
Hysteria doesn't need to make sense. I think that's what OP is suggesting.
In the early/mid 2010s there was a similar hysteria about "deployments per day/week" metric. People were comparing products or teams by saying "My team deploys twice a week", "oh, yeah? we deploy 3 times a day". People viewed the north star of the CD part in CI/CD as pushing to production. The concept was simple, you push your code to main. CIs start churning, deploying it to test environments, then promoting it to staging, the pre-prod, then into blue/green prod deployment, then into prod. All automated, all touchless, happening on every merge, 5-10 times a day. That's engineering efficiency.
Many tried to follow that. It works great if your project is a Jekyll blog of course.
im still not sure what they built software wise that warranted these many commits.
That’s exactly my question reading this. Granted I don’t know much about the training side ofbLLMs, so maybe there’s more code changes involved there than I expect, but in terms of public-facing software, they make, what, a couple chat apps, a product website, and an API?
Ar work, I'm working on some entreprise software which is 50x more complex than anything shipped by Anthropic and we have 2 orders of magnitude less code merged per day.
They are doing something wrong.
I wouldn't be surprised if a good portion of these PRs are multiple agents stuck in a loop constantly overwriting each other changes, with no human involvement at all.
great article. Besides the source control stuff, I found this very interesting (and matches my experience with agentic coding in general, now we'll see until hiring pipelines will take this kind of skillset into account). I am not sure however how easy it is to review 30k PRs/day even if only for intent, and how one can easily notice if the model is taking shortcuts on implementing said intent without reading the code...
---
Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place. This was particularly common in our migration. For example, there were multiple systems that tried to brute force looking up when a change merged to main by linearly reading the commit log instead of asking our code review tool. Anthropic also makes extensive use of adversarial review, and, while this is incredibly effective in many ways, it can also be myopic, so skilled engineers needed to steer Claude away from over-engineering parts of the system in response to hypothetical concerns. For example, one such review was overly defensive about trusting another internal service, which resulted in Claude adding excessive validation and complexity. Our team members still review every change request for the system, but they now examine the intent of the change as opposed to reading every line of code.
This is the single biggest skill of engineering nowadays. Rapidly looking at a solution and saying, "yep that's good keep going" or "nope, that's bad fix this" or even "wrong approach, do X instead"
> I am not sure however how easy it is to review 30k PRs/day even if only for intent, and how one can easily notice if the model is taking shortcuts on implementing said intent without reading the code...
Speed reading code is huge. You just need to focus on certain hotspots and know which things you're scanning for. The traditional approach of reading the code and trying to compile it or reason with it in your head is too slow.
I guess this is also where an adversarial model review of "does this code do what the PR comment says it does" is useful, agree on speed reading being a valuable skill for sure.
I do think that the lgtm gut feel looking at a PR requires quite some years of writing code and reason about it, so not sure how a junior can get that intuitive understanding if they start with agentic coding however.
With 1000 engineers and most PRs bein housekeeping we’re down to 10 prs per day per engineer. And as they said - this is engineers core job now. It’s a new kind of job.
Personally I can easily imagine one or two orders of magnitude more than that. They crossed the rubicon of not gating PRs on tests. In that land you can just roll out changes to production, a small subset of it, during code review to assess behavior. Or during key moments of coding.
I’ve been thinking a lot about this recently: what can we get if every change goes to production in 100ms, and what do we need to do to maintain stability?
As a user, dealing with that kind of churn sounds godawful. I'd pay (or more likely, switch to a competitor) to turn it off.
Nominally, releases gate it
> Earlier in the year, we had largely handwritten the first version of the git-to-Scion ingestion pipeline as a prototype. We thought it was good—until we got our hands on an early internal snapshot of Claude Mythos Preview. We asked it to rewrite the whole thing, and with no further instructions, it produced a version that consumed 20x less memory and was 4x faster, while preserving all of the implicit invariants around ingestion ordering and data structures.
I feel like the advantage of having these models in house can't be understated. Companies with access to inferior models, trying to compete, will be left in the dust when it comes to general development.
Does it really matter since It's one-time import pipeline.
Ripe for optimization against maintainability, of course, but yeah it would seem to matter if that generalizes.
We ran the repo ingestion in a loop for several months while we were split between Scion and git to keep things in sync, so this perf win was actually impactful for us!
Are you already operating with multiple levels of orchestrators and agents? i.e. one human talks to one or more orchestrators who then talks to orchestrators who handles teams of agents?
Yes, this is a common pattern. There is still an opportunity to improve how these agents write code without needlessly conflicting between themselves and we are exploring some ideas.
This is quite insightful: even the best of software engineers (presumably) already are unable to even loosely keep up with these models.
Just a reminder to simply embrace their superior capabilities instead of feeling threatened.
Or they suffer from skill atrophy by overreliance on LLMs.
Others can confirm, but I suspect a lot of inefficiencies in large company programming projects are due to multiple people working on the same code - where you ultimately have to let both above and below average programmers work, and fixing bugs or improving existing implementations is more difficult because of company politics or interactions with others.
> even the best of software engineers are unable to keep up with these models.
It’s more likely they used an earlier version of Claude to write v1, then a later version of Claude rewrote it.
Unless Anthropic engineers were writing code by hand prior to Mythos
There actually was lot more hand-writing of the code back then! Or, at least, only moderately assisted by Claude with a lot of human-still-typing-things-in-an-editor. The ability to delegate larger chunks of code to Claude had a noticeable, step function increase with Mythos Preview.
I don't think anything points to Anthropic having the best software engineers, at least their products surely don't!
Was it a first version in TypeScript then rewrite to rust situation?
The question of what to build rather than how is still central to development. If the article does point to success it's not as much a technical one but of product. That said I'd take these articles with a lot of grains of salt, knowing first-person how corporations spin their successes, including internally.
That's been my thinking lately. The SOTA models are good enough that they "solve" coding but still need directing. The product is still important. Products are human too, and need human attention (still).
Maybe that'll change in the future, but while AI can make awesome images without a good "director" they're pretty cookie-cutter and bland.
Yes, I think we'll find that technical limitations were never the bottleneck. I work for a large corporation and that has been the case here so far. Yes, a lot of things are faster, but issues with product vision, roadmap and cross-team friction aren't gone. They are actually maybe even harder to get right.
Yep, agreed. Like seemingly is the general opinion, taste and direction I think still matter a lot. Efficiency in it of itself is kind of ”solved”, but effectiveness, doing the right things, is not.
Worth mentioning that it’s much easier to point an LLM at something that already exists and optimize it than to create something from scratch.
I’m not saying Claude wouldn’t eventually have done a good job with a from-scratch implementation, but it would have required a lot more hand-holding.
Software is not sold, used, or replaced on speed and memory efficiency. Case in point, most software today (including written by LLMs).
Absolutes matter. If 1x is good enough, making it 20x literally does not matter.
This is true for consumer software. I would imagine for developer tooling and especially internal tools these metrics are rather different?
A 10 minute CI pipeline is great and fine, but everyone absolutely cares if you can bring it down to 30 seconds. This software, by its very nature, is measured by speed.
I used to think this as well, but until it turns out to be a problem for the business worth solving (eg impacting $ machine), it will not get solved. There has to be a why now, otherwise plenty of "problems" get shoved under rug.
Internal tools, IME, are almost always short budget. unless it is in path of revenue, then there is $ and performance matters a great deal *if* more performance means more $.
The Scion project seems like a great example of identifying and addressing scaling issues (just in time, it seems). Just 7 engineers! It makes me think I'd pay Anthropic $300K/year to work on such problems.
But it's a sad example, for open-source and for small engineering companies. Instead of infrastructure being more easily shared with the collective, or valued enough to build a business, now companies can tailor even major infrastructure to their own needs, and avoid all that coordination.
I'm glad they publish their notes to encourage others, but I suspect we're entering an epoch where confidentiality matters most.
Anthropic's problem was merging code at scale, but the scale problem I have with SCM is really merging all the related docs; these get lost in a stream of a forest of comments, PR's, and documentation, all of which (like a wiki) goes rapidly out of date before anyone notices because none of it is automatically correlated.
> Instead of infrastructure being more easily shared with the collective, or valued enough to build a business, now companies can tailor even major infrastructure to their own needs, and avoid all that coordination.
Large companies have always done this. Tons of early 2010s open source projects came out of reimplementing white papers from closed-source Google projects.
It’s also the major downside of this kind of thing. Once someone gets acclimated to how things are done at a large company with custom tools, it’s not always easy to switch to another process, especially a simpler one. There are tons of stories about FAANG engineers joining a startup and being useless.
There’s real value in having consistent tools across an industry. It makes hiring and training easier. Even if you’re convinced that a LLM could crank out a payroll or ordering system that’s perfect for your org, you still have to train the accountants and admin assistants to use it. And you’ve now become the customer support.
The bar is lower with version control, but it’s still more friction than git + GitHub + a well-known workflow.
Excuse me for throwing up this stupid idea, but if code changes per unit of time keep growing at this rate and are not slowing down with infinite merging demand, it seems that this repo will be in a constant state of continuous change and we constantly roll forward with automated code and obviously no direct human supervision, then why even have source control? Why not just let software be fluid and, like Heraclitus river, never step twice in the same repo.
Even with humans out of the loop you will still want features like branches, pull requests, tags/versioning, build reproducibility, etc.
A pull request could be a special case of some other construct?
All the others seem necessary. Maybe you don't need branches at all, of course we do! Things have to have names.
Technically, I guess you could say that a pull request is not a part of a version control system. GitHub has pull requests, but they're not a primitive in git, they're built on top of branches, commits, and merge/rebase. That said, I would expect a version control system built around agentic workflows to have something like it as a core feature.
Feels like a pull request is a proposal for whomever controls the branch ledger, to sign an existing commit as the new head of the branch.
There could be some fun cryptographic ways you could do it directly inside of git.
Tangent: you only really need branches when working on multiple things in parallel, otherwise change IDs are enough
Agents use source control, too. It's a pretty important part of the development process to understand how code has changed.
For example, you can imagine, at the extreme, that agents could get caught in code change loops: a code change seems appropriate, it is implemented, it turns out there's some external factor which makes it not appropriate, the change is reverted, then that same code change seems appropriate again, etc.
Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
In this sentence who would be understanding the code changes, humans?
And which is understanding the code changes meant for?
The agents obviously.. I thought that was obvious.
That's not super reassuring, until I know they can read, remember, and reference reliably at suitable complexity.
Just have the agent rederive the revert, based on the facts of whatever made the change inappropriate.
In theory, an agent could use git-bisect to identify when a bug was introduced. But the operator would have to be more interested in understanding the root cause than in fixing the current form of the bug...
> the operator would have to be more interested in understanding the root cause than in fixing the current form of the bug...
Can you even do the latter without the former?
You can "fix" it (make the current reproducer not reproduce it anymore).
I see it all the time, it is fixing the symptoms usually. It kind of works until it doesn’t and it fails miserably.
So yes you can and it works, sometimes…
For me, Claude regularly looks at git blame after identifying a bug, then reads the commit message and other changes in the commit to understand the context of why that behavior was introduced like that. I wouldn't be surprised if it pulled out a git bisect
Then again, my standard instruction is something like "{bug description}. Find the root cause and propose possible solutions" so I guess I fit your profile
Same, I basically take the derivative of a bug wrt every major dimension before I fix it.
not really an option when you're working on a large codebase where a bisect means that a lot of steps could be multiple-hours of compiles. E.g if on a simple 12-step bisect you get 6/8 with a full rebuild you've already spent an entire work day on this
> Agents use source control, too.
This is true, but at 30,000 merges a day we're talking about nearly 1 million merges a month, and purely autonomous agentic merges tend to be noisy. Is this still a tractable problem, even for an agent? Do revisions older than a few days, other than perhaps release snapshots, have any relevance for such a process?
Not all merges are equal, even if it's a monorepo, it's probably rare that you'd have to look at the whole codebase to implement a new feature or find the root cause of a bug. Once you start looking at the history of a specific folder, it becomes more tractable.
Of course that does require some folder structure, and agents are not necessarily great at that, but they're still decent enough that you do end up with many folders not being impacted hour or even daily.
I am not sure that I can agree with this at this scale, especially for the hotspots in the code. But I admit that I am speculating here.
That won't last. I see that as a disaster waiting to happen. There is absolutely no way they have control over what gets produced at that rate.
> There is absolutely no way they have control over what gets produced at that rate.
I'd wage that volume of changes is not leading to forward progress, but is a clear sign of code churn.
This in turn is a clear tell they are dealing with spaghetti code.
> spaghetti code wow, that's a term i haven't heard in a minute. almost makes me nostalgic for all the human-made slop we had to deal with
At least spaghetti code had one saving grace: whoever wrote it could only make so much of it before they gave up.
> Agents use source control, too. It's a pretty important part of the development process to understand how code has changed.
This is a great idea, now they just have to use/do it. Maybe they can use it effectively by cloning the repo and having all the history at hand, rather than continuously scraping web pages with git content formatted for human brwosing.
You're referring to a completely separate system that, for all we know, might not even be run by AI companies.
I’m not advocating for no source control, but you can avoid loops like this with Architectural Decision Records. If you’re real lazy, let the agent maintain the ADRs and they will usually call out when these “painted into a corner” scenarios happen and you can then reason them around it.
Right. But how to prevent adversarial agents working on the same code base from treating the ADR list as a form of malicious message exchange?
Don’t let adversarial agents work on your code?? What is protecting an ADR against adversarial agents going to solve if they can just embed a highly advanced, obfuscated ‘rm -rf windows32’ anyway
Why give an adversarial agent write access to your code
> Hmm, I have destroyed the file again, that's odd. Let me rebuild it from a half recollection of what it was supposed to do based on my compacted context.
It's the right approach if your business is selling tokens and the customer doesn't know any better.
You're absolutely right, this mistake is unacceptable. I take full responsibility for this.
Anthropic can just replace pesky source control with a much better source guardrails. Aren't they all the rage and next step evolution in the computer security? Just ask Claud "pretty please do version control" and that should be enough, right?:)
You absolutely know there is one of those AI bros out there in social media posting on how they feel euphoric because their agents don't speak git anymore and just pass a zip with the source code around to other agents as handoffs.
>Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
most of the harnesses are checkpointing most of the data of a git commit aside from the formal release and messaging, anyway; thats why they all have a similar problem with SQL dbs exploding in size after a bit of time.
Right now we're in this weird transition where most of the harness systems are layering git tool use w/ the constant snapshotting AND a memory subsystem often, probably leaves a lot on the table in saving wasted token use to just ditch the vcs all together and rely on the already persistent and existent checkpoints and memories for reconstruction.
> Obviously you can prevent that from occurring with some extra infrastructure in-place, but at that point you're basically creating source control.
Or code comments which they are already doing a lot of. I used to get annoyed by their verbose comments but at this point I've seen them help a few times and it's not like I'm actually maintaining the code myself anymore so it doesn't really bother me
With the way they are shipping code, source control history has no practical value. If barely any code older than 5 days still exists in the codebase, and nobody reads the commit logs, you can throw out most things in a source control system. It is basically a database of latest version of code, that's about it.
I expect mainly because the coding models are trained to roleplay that process and don't work as well if they can't pretend to use git
I've been doing that for one project/experiment. No source control. Tests and an agent swarm let loose.
It works, but there is a significant amount of churn. I find when the swarms use git it's less churn and less token usage.
Yep. You need some amount of workspace separation and reconciliation.
Lets take the idea further, and since nobody seems to be reading it, just throw away the source code. Just let the agents go directly to assembly.
Does your agent not know how to just build a binary?
If not, agents can reverse engineer binaries.
I’ve been building and playing with this idea for a while. Mostly with simple chips and simple programs. It seems like a good idea on the surface, but unless they build the whole OS the ordeal gets much more difficult. There is also a lot of work building / finding training data without decompiling and labelling current exes.
Assembly is source code...
Can confirm: In the 1970s our high school had a minicomputer for teaching programming, and it was turned off when not in use. After turning it on, we would enter the magic three binary instructions into memory using sixteen toggles on the front panel.
Give it six months and there'll be have Optimus robots toggling switches to load terrabytes of binary programs directly into memory. Nobody wrote it, nobody will read it, but it will run everything.
The Machine Stops: https://www.cs.ucdavis.edu/~koehl/Teaching/ECS188/PDF_files/...
>Can confirm: After turning it on, we would enter the magic three binary instructions
that's not quite confirmation that assembly is source code
The point is that if we can put binary directly into RAM, then what is assembly if not a programming language? Assembly is source code in a programming language that is turned into binary by a compiler called an assembler.
If you don’t feel that toggling the binary 16-bit words into RAM demonstrates that binary is what actually runs on the hardware, and that assembly is source code for a binary, well ok. I find it persuasive.
I think we can take this further, let's just end the human race as we know since we've obviously just successfully completed our technology tree. There is nothing left for us to do anymore! It was fun guys. Well, it really wasn't. But we made a lot of money! OK, maybe not everyone. But some people did! And that's all that matters. Top of the leaderboard!
I can get you a million followers for this idea by 4 PM :-)
Source code exists, that’s not in doubt. If it exists it needs to be managed.
From the post (especially the end) it seems that they are essentially letting software be fluid.
Reading this post put me in a similar head space.
I believe you're mostly thinking of version history though. There’s plenty of other reasons to have source control we still need, like guarding against racing writes or continuous integration to ensure correctness. Even some industries require version history for compliance reasons.
I do love how AI is challenging us to rethink our tools though rather than blindly reaching to git for every project.
I don't think this is stupid, and I think the logical extension to this is agents will start writing directly to prod with fully automated code changes.
Pulling that off successfully at scale would be pretty incredible to watch.
Your last statement holds even after removing the word "successfully".
I let my Claude write to prod in some projects. But after comitting, pushing, waiting for CI to run unit tests and build the docker image. Then it can deploy the image on k8s. Sure, it can also just run commands on the pod. But the agent benefits from the ability to rollback, identify at a glance which version is running, what has changed since then and why
We built all this tooling because it's useful. If you replace the human with an agent the tooling doesn't stop being useful. If anything it becomes more useful, since AI Agents excell at reading tool output but struggle with maintaining long-term state
That's because we're in a world where the human and agent make mistakes. But think about the world where the agent doesn't make any mistakes anymore and doesn't need to rollback or figure out what has changed.
So agent would have reached perfection. I'm not sure humans can create a machine capable of perfection. This is a philosophical question waiting for a definitive answer.
I don't think we will reach a world where agents make no mistakes. Because there are always tradeoffs. So an agent could choose the "wrong" tradeoff and we call that a "mistake", but is that really a "mistake"? Arguably not
If the agent is able to make fewer mistakes but chose the wrong tradeoff it will be faster to fix forward than to look back, figure out why it happened, dig out the rollback, and then figure out how to change the decision.
No it won’t.
Or humans just prompt wrong thing or people actually don’t like feature.
Lisp machines finally vindicated. Took only 40 years and a historic paradigm shift
Good way to keep metrics and prove work is happening would be my guess.
They are probably like 3-6 months away from just letting their agents work directly on prod after realising that repos don't scale enough.
didnt the article say thay engineers still review change requests? They just switched from reviewing code to reviewing intent.
So "no direct human supervision" seems like a misunderstanding. Article even bigs up the purpose of engineers with a hilarious (to me at least) line about auto adversial reviews resulting in over engineered designs.
Good point, I wasn't clear enough about my thoughts here. I tried to say that in a multiple commits/s world the review might not be direct line/hunk review of the code but a more "indirect", abstracted or higher level human oversight, which is how I understand an intent review is.
I'm not saying that Anthropic here close their eyes and tell Claude to ship and that in the future there will be no code review at all, but that if we have 30K merges/day, human review seems to need to be done at some higher level, likely AI assisted like it seems to be done with their Antique code review tool mentioned in the post.
I mean i’m solodeving and on my github 30 commits a day is light green. Record probably was 280/day, but I’m not sure if every commit is merge to them. or if they just count PRs
> then why even have source control?
I know you are joking, but the answer is the same. Because the agents need it, not because any human need it. The Agents quite actively use the history. It's more effective than any comments or separate documentation might be for the agents working on the code.
There would be so much noise in it, the history is going to be pretty much useless.
Quite the claim to present without evidence
Yet, in all that noise, a git bisect will still function to find when a bug was introduced.
Do the math yourself: A bug is picked up 2 week after commit. In the meantime, a good 4096+ commits have happened since. You are looking for the pathological case, at +- 12 commits.
Do you think each and every one of those are going to be "good" commits, commits that still "compile" or function. Possible, but unlikely given the insane velocity.
I mean, if you have sufficiently well developed CI, all of those commits will 100% function. That's a part of even allowing the commit to go through.
My last vibe coded project ended up with over 250 unit tests, without any prompting for it. These new models are VERY test heavy. I'm sure that's the reason why scores are so much better than they used to be.
You should read the article
Many parts of the Git distributed version control are the results of good decision made. Anthropic source control team just think that their trade-offs needs are vastly different from other businesses. They said these too:
1. they have no-cap token spend allowance internally
2. their agent swarms are amongst the largest if not the largest, again because #1 above.
3. their git repo is enormously large (100GB uncompressed git history on main, 25GB compressed).
4. their merges per day count grew an order of magnitude from May to September to 30000 times per day, they are not seeing it slowing down anytime and probably only going to accelerate.
5. all of their code agents are networked and has network access, full local pull down is not needed at all and too slow (see #3)
6. their design of the Scion is based off prior art Google’s Piper and Meta’s Sapling.
Given their own constraints and characteristics of their own. It only makes sense to build their own homegrown solution based on prior arts. But this was NOT a complete redesign from scratch - so it's an full online download only what you need type of source control system. The problem is the version control software itself but its prior tradeoffs are misaligned with their scaling needs.
PS, if anyone noticed, Anthropic starts to concealing their employee full names and replaced with only first name + last name initiate, not sure why.
PPS, the project Scion is so bad, scn is worst, it reminds people of prior SVN, which is bad by itself.
> 100GB uncompressed git history on main, 25GB
I might be completely out of context, but why is it only 4x compression? Do they commit a lot of binaries? I just don't understand why would code compress so badly.
I have no special information, but it could just be Amdahl's Law. If one guy (or bot) checked in 5GB of binaries before you thought to enable git-lfs, then that's not a 100/25 ratio, it's a 95/20 ratio. And if it was 10GB, that's a 90/15 ratio, and so on. IIUC, the old un-LFS'ed binaries remain in the history unless you actively `git lfs migrate` and rewrite your whole history, which they might not want (or care enough) to do.
Creating an entirely new version control system and switching to that is supposed to be less disruptive than rewriting history?
But Piper is a reimplementation of perforce which makes me wonder why they couldn't just license it. Why does it make sense to do this. Perforce should scale to the level they're talking about. Google were using it even when they had many tens of thousands of developers and multiple changesets per second and presumably it got better since then
In the age of AI writing code, maybe the question flips from "why not license it" to "why not reimplement it"?
Is it really their comparative advantage? Perforce can use Claude too but can amortize the cost over many customers so it should be cheaper and faster to reuse.
I mean 30k CLs per day isn't much. That's one every few seconds. Well within what a decent perforce setup can do with enough replicas.
You loose the free promo for your LLM, though.
Perforce can amortize the cost of Claude, but Anthropic can just use idle Claude-hours for free. They're already renting the datacenter capacity.
We can assume there is some price that would clear those Claude hours, especially as all the labs seem to constantly struggle with overload. So it's not free, there is opportunity cost to using Claude themselves.
Google implemented Piper because Perforce simply didn't scale well. In the years since, P4 has barely improved at all. Licensing P4 would not help and Google doesn't offer Piper licenses.
I feel like "didn't scale well" is unfair to Perforce. It scaled to accommodate an absurd monorepo that might have been, at the moment they switched to Piper, the largest single software project in history. It scaled to handle the needs of every organization in the industry except Google and Microsoft.
It scaled pretty well compared to everything else. I was there, they were running tens of thousands of engineers and hundreds of p4 commands per second off a 16 core box with a few replicas before starting the Piper migration. Commit rates were comparable to what Anthropic report. And although you say it barely improved, they were doing that sort of scale on 15 year old hardware and the p4 admins wrote a paper where they said Perforce had improved scalability a lot over time, so it's not like the software was static. Architecturally Scion sounds rather similar.
It's always a question of opportunity cost. The culture that led to Piper also led to them creating their own IDEs and everything else, often with questionable outcomes. Google had the profitability to get away with that kind of profligacy but do the AI labs really want to go the same route?
These productivity gains must go somewhere.
If you're employing 10,000 people and CodeSearch makes them all 2% more efficient, then why not do it? I think Google made their money back easily on that investment.
Cider and Cider-V were both forks of existing IDEs with Google-specific stuff wedged in. They weren't greenfield projects.
Blaze was absolutely worth it. Nothing else comes close.
I'm sure not _every_ tooling project was a win, but I don't think it's fair to frame it as "every Cider engineer could have been ramping up remarketing and advertising instead." Google's early days involved trying everything and keeping what worked. The tooling improvements worked.
Before the Piper switchover, Google’s Perforce instance was running on one of the most expensive PCs ever built, with over $1m of DRAM. Its centralized design was not a setup one would want to copy.
Yes but that was 15 years ago and do you really think the lifetime cost of building and running a custom VCS will be less than $1m, all in?
Also IIRC from the original 2011 Perforce admins paper before Piper, they said they had seriously overspecced the RAM and could use a lot less. There's an annotated copy here: https://www.anantjain.xyz/posts/perforce
Anthropic is protective of employee names because of the sheer amount of hacking/phishing attempts that happen. A friend of mine got a job at Anthropic and within a week he was getting linkedin and email spam from people impersonating Dario.
So? That's true of any employee at any company worth more than a billy and that's not a small amount of companies. I have gotten personal text messages from the CEO of my company, emails, and fake LI invites all at companies nobody has ever heard of.
re: PPS: Employees are an attack vector. Making their identities harder to discover is a protection.
Gives me a vaguely creepy feeling, but it's reasonable.
Perhaps I'm dense but I don't see how any of that answers the question "why even have source control?"
I think the answer is that they don't, indeed, need source control but just a way to let their agents resolve editing conflicts. It's basically yet another message board.
> Many parts of the Git distributed version control are the results of good decision made.
For how the Linux Kernel, a highly decentralized, completely open source project, with a single monotonic product, tyrannically lead (a good thing) - is developed.
Permissions, Partial Checkouts, Sub Repos, Vendoring, pub/sub... How many companies bought into git and then had to buy more tooling (like artifactory) because their 3rd party dependency's needed a home. Thats a feature that git does not support because git, as designed, did not need it.
Think about how much better your life would be if every time you typed npm,pip,goget,cargo it fetched THROUGH your repo. If those updates triggered code reviews of whatever changes you're going to consume. You are code reviewing those updates aren't you (someone is, cause supply chain attacks don't last that long)
Git has some weak points, Google doesn't use it for a reason. Epic created lore for a reason, This exists for a reason. Im not sure that any of them are the next thing that replaces GIT but it's getting to be time. Tools like this are going to keep popping up till someone gets it right.
All of this is unsustainable in short, not to mention medium term (not just for affordability, even just security reasons) so I don't think there is a reason to look into scaling this mess.
Not using the full names is a mitigation for talent poaching.
There are orgs that are moving to not including names at all as the next step on that trajectory.
Apple has always been there.
Not always. Before Jobs II, the apps would have credits in the about section.
> Given their own constraints and characteristics of their own. It only makes sense to build their own homegrown solution based on prior arts.
Does it, though?
What problem do they actually have that requires redesigning a whole new vcs?
Merge conflicts? That's not exactly a vcs problem but a "we changed the same code at the same time" problem. No tooling on earth eliminates the need to verify conflict resolutions if agent A renames int foo to int bar while agent B renames int foo to int baz.
Growing repo size? They should take a long hard look at what they are doing. How is their software architected? Is every single little change and half-baked AI churn commit pushed to the main repository, or does the main repo only receive the deliverable? Is everything under the sun tracked in a mono repo or are projects stored independently?
Do they take any precaution at identifying churn and taking steps to minimize it?
Frankly, this reads like a bullshit PR stunt, much like the bullshit curl vulnerability findings. They have a self-inflicted problem, and their solution is to persist the problem and mitigate the problems by rolling out ad-hoc tooling? It reads like "if all you have is a hammer" kind of situation.
This has always been the dream, code flowing constantly like a river, like green rain. Imagine a world where you never use the same piece of software twice, because every day it changes.
This is how we solve cybersecurity, by making code into constant shifting sand, so a vulnerability never truly lives long enough to be exploited at scale. Sure, a hacker may occasionally pick out a vulnerability like a bear picks salmon out of a river, but it makes larger scale cyberattacks impractical. The code is a firehose, constantly in flux.
We could do all of this right now. I see it laid out like a great tapestry, as specs are reimplemented over and over again, in an endless sea of meetings. Some in Rust but others in PHP or Cold Fusion. Entire lives conjoured from the ether to simulate stan so he could be a TPM on the multibusiness unit unified receivables, this time in vb.net and with 3 kids instead of two.
Due to the pace of app changes, people on reddit seem to think Spotify is a constant "green rain"
Correct me if I'm wrong, but I don't think this has ever been the dream? The dream was to write software once, and then use it forever.
The hackers have AIs too.
This doesn't make everything secure. It turns the universe into what we see in Star Trek, where everything is hackable all the time by everyone just by trying hard enough, or, in the real world, spending enough. A lot of the utility of hacking systems can be obtained in running off with the data so the inability to use a break tomorrow is not necessarily a problem, especially if I just ask my magic code fairy for another break tomorrow and the remote system has, to a first approximation, learned nothing from the previous breach as it busily churns away.
This is not the way. In fact for secure software I'd happily testify in court as an expert witness that this would be criminal negligence.
As for how closely it matches what Anthropic is doing, no comment. I'd need a lot of time to look at that and decide. That they can write code easily I do not question, whether they have 100GB of problems or 2GB of problem and 98GB of garbage that is slowing down even their AIs is not something we can get from a happy PR puff piece. I'm just speaking to the idea of letting code mutate wildly and constantly.
You’d expect that while the frontier of changes fans out and grows, some of them burn out and disappear and some of them survive and solidify into a relatively static code base.
The big baddies of build have worked this way for a long time already. Git surfaces a single Merkle tree id for the whole repo on each commit, but build tools like bazel/blaze, buck, pants, et al look at hashes of subtrees and use cached artifacts for the parts of the tree that didn’t change.
Your CI no longer becomes a binary signal on the code and more of a continuous signal on how much your “this works” pointer has caught up with your “this is the latest code” pointer.
My mental model, for what it's worth, is this: - deterministic foundation - non-deterministic on top of it operating against (and building) it
That is, the deterministic part is the platform that is being built and that non-deterministic agents will use and build it, but that layer is a key part of how they do work and will continue to be.
https://youtu.be/TdTMeNXCnTs
Have the same idea. Compiled code with static type checking, existing SAST and formatting tools and there is quite sensible code hallucinated. You still have to check higher level requirements on your own … but LLMs with all that was mentioned are hard to beat on code quality. I usually had devs oposing those tools “because it makes their work harder” and “why can’t they just do stuff” - LLM is not like that ;) LLM doesn’t waste 2 weeks on new project arguing between agents weather it should be tabs or spaces for indentation.
Funny you mention that, that’s actually the direction I’m taking with the architecture that I’m building
In fact most of the architecture work I’m building, assumes that there will not be a human reading code, they read (effectively) UML and data diagrams
so most of the interesting parts I’m working on now are how do you make code blazingly fast, formally verified and memory safe if you remove the constraint of human interpretability of machine instruction
You're asking if the bitter lesson is true and the neural nets will become so general purpose, they'll eat the source control as well? Yes, but that won't happen for another couple months.
You think CI and distributed builds aren't necessary if you change the code fast enough?
There's lots to improve in how we handle code, and losing the agreement on what the codebase even is sounds like an awful regression.
Source control has fine-grained metadata that is invaluable for LLMs.
They can just analyze PRs and diffs without loading the entire 10,000 line file.
Also, anti-AI folks really have a fundamental misunderstanding of how everything works in the LLM world and is probably why all their predictions are wrong
I have a theory (loosely held, it's just an idea, not a belief) that this could lead to something I call Evolutionary Architecture. That instead of waiting for vulnerabilities to be discovered in code, that systems could end up with code being changed and re-implemented by agents so fast that attacks aren't able to probe for attacks because they end up probing code that by the time the answer is come it, in production that code no longer exists because it was moved to another language or something. The change to prevent vulnerabilities isn't the reason for change, but the change has defensive side effects.
All of this may be slightly inspired by the Militech neural matrix in Cyberpunk 2077
I've been developing a weaker version of this line of thought for code reviews for large architectural features.
A big motivator of thinking architecture through really well is that although the software may work just fine now, a good architecture will make it easy to add a feature a year from now, whereas a bad architecture will necessitate significant rewriting.
But significant rewriting is cheap nowadays, so in code reviews I put a lot more weight on whether it works today, and a lot less weight on whether we'll need to change it when making a different change.
Entire classes of things like "should this be in this component or split out into its own" just don't matter anymore because no matter how tedious changing that might be once there's a ton of things depending on it, that's the kind of thing AI can do no problem.
It seems those classes of things are more important than ever. It's not about how tedious it is - it's about the models being able to build accurate context and understanding. All the same things that made it better for humans also makes it better for models. More specifically in this context, well-maintained code results in fewer bugs and lower token usage.
I tend in the other direction when I can (for components and microservices - harder for monorepos): I encourage refactors on a continual basis, as it prevents the models from getting bogged down with the way the system used to behave across changes.
I think this works well until you get into data, state, history, external interfaces etc. Large scale data migrations are still risky, anything with external consumers you can't quickly change is risky, etc. Internals inside a service I agree, yeah you can let it slide a lot.
I also found though, with agents you can quickly create large amounts of code, when you go to refactor / fix some design aspect that touches large parts of the system this consumes a lot of tokens and actually still takes meaningful time.
It's like, with a digger you can quickly create a hole! Digger can quickly clean up the work of 10 people with shovels! But if you use a digger for 1 week and you were putting the dirt in the wrong place, you find you have a HUGE pile of dirt to move and will still take a long time to clean up.
Directionally I agree with you, but I think design still matters.
AI can do it but will cost a lot more tokens than if the code is architected for easy changes.
"Security by Obscurity" is being replaced by "Security by Velocity"?
The Military has the "Survivability Onion", Security through Obscurity is the "Don't be seen" layer, while "Security by Velocity" would be the "Don't be acquired" layer.
This is pretty close to Moving Target Defense (MTD) and Binary Diversification.
TIL! Close, just taken up a notch that wasn't feasible before where MTD was more configuration focused. As CPUs are to FPGAs, MTD would be to this. Rather than telling the config to change on the fly, the code changes on they fly. And rather than having multiple Binaries prepared, the pipeline of changing binaries are more just-in-time than batched.
Maybe a frequent checkpoint suffices ? Let’s say I have hourly snapshots. In case of an outage I can restore the earliest checkpoint things were working well and have LLMs triage what the root cause is since then. Some code changes will be irrelevant. Sure, but reading code is now basically free ?
Once you have a single main history of checkpoints and want to resolve conflicts, you're likely already dealing with all the hard parts of getting high throughput
No conflicts. Just checkpoints of all the prod deployed code. At some point it was working. That is the one you start from.
The one you started from is already behind by the time you're done with the change. You either deal with conflicts or limit work to a single agent.
I don’t understand.
At 00:00:00 utc you take a snapshot of the code deployed to all your machines (if different code was deployed in different machines then it is part of this capture).
That is the unambiguous code state of your production system.
There is no conflict in a prod state by definition. If there was it cannot be prod.
Because you need to be able to revert. It's not rare at all for agents to get wedged to such a degree that you have to revert and start over again.
To me this sounds like an awfully stupid idea because it's a software engineering red flag. Code is supposedly designed around the need to simplify introducing changes where changes need to be introduced more often. One dimension of this requirement is minimize merge conflicts, and limit those to when you need to absolutely change the exact line of code some other commit is changing.
If your code changes are all over the place, touching half the repo and stepping on everyone's toes, your problem is not the version control system. Your problem is your changes, and the mess you are leaving in your wake.
Because you still need short-lived branches for editing, CI/CD gates, bisecting, and an audit trail.
Then how do you have multiple independent agents working on the same repo?
> it seems that this repo will be in a constant state of continuous change and we constantly roll forward with automated code and obviously no direct human supervision
The whole thing, including ingestion and review, is not meant to be used by humans.
This is the shared memory / history of a horde of coding agents.
Is this really something to be proud of? I make heavy use of AI, but for anything that is eventually going to production there's still a human reviewer. Based on this I assume Anthropic is shipping code with no human review? Is the future of our industry going to be oneshotting prompts with out manual code review and then having AI fix the bugs found in production buy customers? I'm not trying to be a Curmudgeon, but am I missing something?
If AI can improve/speed up writing code, why shouldn't it improve/speed up code review?
You can probably still find the leaked code online. It’s not exactly a bastion of software engineering. Plenty of folks have reviewed it deeply.
Where do you draw the line, and where does someone take final responsibility? I don't have an issue with allowing AI to do everything with limited to no human verification for hobby projects. I have an issue with allowing AI to write and ship medical device software with no human review. What's the middle ground?
You still take the full responsibility, I don't think leveraging AI reviews removes any layer of responsibility. The challenge is building the feedback loop (or the agent that builds the feedback loop) to make the review process actually catch what it needs to catch.
I think the line is wherever you feel comfortable with pushing the changes yourself. You feel like adequate testing had been done, basing it on previous testing periods or how successful AI runs have been.
Whether you or AI wrote the code, you decide when it's safe enough to put your name on it. Like it's always been!
If AI does something good, you take responsibility. If it does something bad - "the agent messed up". /s
They review based on the intent of the change not jz blindly.. I think it's mentioned in the article where the job of a developer is to provide high level design decisions and saying no to claude so that it doesn't over engineer solutions.. As the demand for good, but not perfect, software is the bottleneck for anthropic and access to better internal models, I feel that's good enough for their workflow as they've started repeatedly that this is just suitable only for them and they want to share their knowledge like a good software company does.. you talk as though code reviews were not a joke in lots of companies where only specific teams/companies does detailed reviews.. many teams/companies jz blindly approve based on tests passing or just do a high level overview check before merging..
I must have been lucky in my career then, in my experience code review was much more then a five minute check followed by approval.
If your customers accept it, why not? It's much cheaper and testing in production is not really new.
Somehow, those principles remind me of SVN (the version control that was popular before git). The central server and clients can't run independently... Does not feel like progress.
I wonder if internally they still have their agents writing nonsensical long essays for PR descriptions. We've yet to figure that out inhouse...
Based on the features and design, did Anthropic just re-invent and vibe-code Subversion?
I had the same thought. I think they did.
Anthropic had an urgent problem, which is why they have reverted to a quick hack of centralisation. However there is a more important goal which will swing them back to decentralization and federation: they need isolation.
All the evidence of AI hacking events points to the need for agent source control and collaboration in completely controlled environments, which precludes giving access to the one-repo-to-rule-them-all in an AI-coded VCS.
I don't get one thing, if agents are so great at developing software at massive scale, why wouldn't Anthropic, instead of just selling tokens to other companies to use its agents, let those agents develop highly profitable services for Anthropic to support their IPO valuation or not require raising money at all? For example, a competitor to Google search.
Anthropic already has their own WebSearch tool in Claude
In a gold rush you want to sell the shovels, and in this case the AI is the shovel.
I want to challenge that metaphor a little bit and ask the question: who did sell the shovels in the California gold rush? Can we name a company, or a person, who amassed great wealth and fame for doing it? Can we name a company that survives until 2026, still selling shovels or excavation equipment, still famous for their big break in 1849? I don't think so.
However, I can tell you someone who fits all those criteria: Levi Strauss. Strauss founded a company to sell dry goods, as well as tents and jeans, and eventually specialized in those denim jeans. Levi Strauss became a household name and an enduring success of the Gold Rush.
Therefore, I say that this motto is flawed; so let's correct it: "In a gold rush, sell the work clothes!"
The person who most famously got rich selling shovels in the Gold Rush was Samuel Brannan. You'll find the street named after him just a short walk from both the Anthropic and OpenAI HQs.
My take on picks and shovels was always a more cynical one. The beauty of the metaphor comes from the fact that after the miners went bust and left their picks and shovels where they lay, they could be collected and sold again to the next dreamer.
Levis.
^"All comments by this account herein are generated by a neural network, unless otherwise specified."
Bruhhh. My bruh. With 15,541 karma, and you are as ignorant as you are petty.
You’ve desperately tried to call me out and flag me at least 3 times now.
You do not know what a neural network is? Do you know what a neuron is? Do you… possess any neurons?
https://m.xkcd.com/2173/
On a forum where the most favored pastime is to call out stuff you don't like as “slop”, (a flaggable ad hominem) you are now taking pride in outing me somehow, as someone who describes exactly what I'm doing, publicly in my profile?
And in the process, you (and apparently, an entire flag-it brigade of ignorant motherfuckers) have publicly fallen for the most transparent and avoidable trolls in cog. sci. culture. Congratulations, mate!
I believe that you owe me an apology, along with a public retraction, and an unflagging of all the previous comments that you have falsely flagged on false pretenses and ignorant, petty, vindictive motives.
I would have been wary if I saw that message on your bio too. Don't be too quick to dismiss what they said as purely ignorant, we are in weird times.
Are you asserting that you are a biological neural network? Because as I'm sure you're well aware "neural network" is a jargon term for a particular software architecture that contains zero neurons.
Your output is both verbose and tendentious in a way that suggests artificiality. If I'm wrong, I apologise and I will unflag.
Jesus fucking Christ
I literally said this out loud when I saw that comment.
Because creating a successful software product is more than just writing code.
If investment managers are so good at investing why don’t they just make money for themselves instead of trying to get people to invest with them?
Investors have a sound economic reason to diversify but your argument works great for pundits.
Some do exactly that?
cause they literally do that? they invest their own money, and then they ask people for money to invest for them so they can get a cut out of it too?
The ones who are actually good at investing largely do. The vast majority of investment managers soliciting new accounts are pure sales roles, not investors. Their job is to get as many assets under management as possible; that's how they get paid. They will fluff up the presentation with some nice charts and stuff but the actual "investing" will just be tossing the money into some generic funds.
Well that's actually true though. The very very few investors that are good at investing do invest their own money. The vast majority of investors are no better than index funds.
> If investment managers are so good at investing why don’t they just make money for themselves instead of trying to get people to invest with them?
The good ones do.
Citadel Securities, Renaissance, Jane Street, HRT, Jump, consistently do 50%+ annual returns over the past 15-20 years. They obviously don’t accept outside investor money.
Now, hedge funds that accept outside money like DE Shaw, Bridgewater and shit, yeah, they barely outperform S&P500 after their fees.
Then at the bottom, you have financial advisors who underperform S&P500 after their fees, but these advisors provide emotional support, so that’s what you’re paying for.
> consistently do 50%+ annual returns over the past 15-20 years
That is only if you consider high-frequency trading investing. Which is very hard to, because on traditional investing timescales HFT is delta-neutral.
> The good ones do.
I mean, there isn't any evidence other than heresay.
> I mean, there isn't any evidence other than heresay
3x leverage on VTI/VOO yields 20% annual return over the past 30 years.
Even you, a passive index fund investor, can do this in your Schwab/Fidelity account.
Now consider algorithmic prop trading firms with 10x, 20x leverage. You do the math.
For reference, most people take 5x leverage on their house (20% down payment).
You think Citadel and Jane Street are fairy tales? The comment you replied to gave plenty of very pointed evidence. Even "bad" trading firms are disappointed to make 20% returns year over year...
Leverage.
Because fees as a manager of someone else’s money can be higher than the full amount of the returns if it were just your own money.
Fees on someone else’s capital: 1% of 20% of 1bn = 2m
Returns on your own capital: 20% of 1m = 200k
Because they are not good at investing.
> For example, a competitor to Google search
What do you think Claude (chatbot web interface) is?
What do you think it uses to search to web?
I don't know, but it's a good question? I have a bunch of test questions that Claude is unable to solve using the web, any version, Fable 5.1 to Haiku 5.5, and pretty much every OpenAI model gets it easily.
My only explanation is that Anthropic's search backend sucks.
Google, to find articles generated by claude, of course!
Most of the time If I ask it about a current event, it won't actually search the web until I tell it to. It will just give me outdated information in its training data, extremely confidently.
I use the ChatGPT general purpose interface rather than Claude, and this is not at all my experience. ChatGPT will almost always do one or more web searches for any query about recent or historical events.
Writing code is less than 10% of the average business.
One word: focus.
> a competitor to Google search
Fielding a web index takes a massive amount of compute and storage of a very different kind than what Anthropic is using. AI is extremely heavy on GPU, maybe somewhat on storage but (comparatively) light on CPU. They need CPU to train coding agents, obviously, but scaling to a billion user web search is a whole different thing.
You have no idea. They have the entire internet and everything else that’s ever been written anywhere they could get their hands on. I highly doubt they use Google internally for their web searches. Get a Let’s Encrypt certificate for any website that isn’t published anywhere else. The Anthropic and OpenAI bots are among the first to read it.
what do you think they train their ai on?
Lack of focus can kill you, too. OpenAI learnt this the hard way: https://www.wired.com/story/openai-shuts-down-sora-ipo-ai-su...
I was upset at that for a bit. And they removed Group Chats too.
Maybe that's their way of becoming Google?
Lack of focus, like building your own SCM
it takes orders of magnitude more focus to target "Market: World" vs targeting "Market: my company's employees". For example in enterprise software, you need sales people. for internal software, usually the dev build it is fine for the job.
But the article clearly describes how this SCM is just instrumental in their goal to build and serve better models. It's not some unrelated product they hope to sell.
No it doesn’t, it doesn’t even talk about what they’re building
There is more money in selling shovels than you can ever win by having a monopoly on really effective shovels. They don't need to build a competitor to Google there's 300 companies throwing money at them while they build the "next google" and even if they all fail Anthropic still wins.
Because creating a successful company takes a lot more than writing code.
Anthropic could easily build a CRM equivalent to or better than salesforce but they still wouldn’t be able to compete with salesforce. It makes a lot more sense to sell salesforce tokens instead.
Why would they not be able to compete with salesforce? This isn't some tiny startup. This is one of the most deep pocketed companies in the world. Anthropic has announced they are about to spend 5 salesforces worth of money on cloud expenditures in the next couple years. They are projected to be valued at about 10 salesforces after IPO.
Imagine how much money they'd raise if they just said one day "we are going after salesforce bottom line now." It would be like chumming the water.
Salesforce is a part owner in Anthropic, they are not separate entities who would compete.
That scales to 1-2 ideas. It does not scale to "we want to run EVERY company with AI". They'd either need to scale out to every single sector at the same time, or they will lose their competitive advantage as the competition makes their own AI (it's not a secret how it works, at this point it's more about investment).
Selling AI to everyone does scale though!
> Because creating a successful company takes a lot more than writing code.
As many in the industry say now - writing code was never the hard part
If farmers are so good at farming, why don't they grow every crop there is and just live for free instead of selling 1-2 crops?
That’s kinda like asking why does any company make and sell any tool rather than just using the tool to make money? Why does Milwaukee sell power tools to contractors rather than keeping those tools to themselves to profit from contracting work?
Because that company does not mint (supposedly) thinking machines - the lack of humans at a price point that is feasible is often the reason - it would be more profitable to sell the tooling because its impossible to maintain all the layers of specialization within one company.
It's the same for software companies too, they do much more than just producing code, it wouldn't be feasible for Anthropic to do all that other stuff too.
Eh but if Milwaukee was to then invent a power tool that it claimed could soon do the contractors' whole job... OPs question would correspondingly exist there too.
A student of history might note that power tools do indeed allow a worker to replace multiple workers not equipped with power tools.
Not the same though, equivalent would be a tool, rented from Milwaukee, that does all the work with minimal input from contractor .
Well both Microsoft and Google are part owners of Anthropic, and Anthropic uses their data centers. I think those companies would use that leverage to prevent Anthropic from directly competing with them
Why would they? They would stand to profit since anthropic would have to take theor market share by doing the same for cheaper.
Probably because the shovel Anthropic is selling may not be as good as they claim it to be.
Same reason banks sell mortgages rather than just buying all the homes.
Ha
Shush now, don’t question our value proposition
> 30k merges per day
> 25 million CI checkouts
833 checkouts for every merge? That's quite a few. Even accounting for multiple environments at different stages of production readiness, that's pretty weird.
How about 100GB disk space for a checkout. That's pretty weird too. Makes you wonder if they're the best stewards for code. Makes you wonder about turning claude loose on your codebase.
I personally haven't dealt with a big-corp level of mono-repo. The choices you make might be justifiable to bring in certain binaries as opposed to resolve artifacts from some type of library repo. Especially if they are doing web front ends and whatnot?
And actually, the lack of an artifact server and some sort of proper knowledge of compatibility requiring a CI pull across hundreds of different projects with every merge is probably the explanation. Seems like there is an innovation waiting to happen there.
For local projects I'm using a repo per module/package, creating tagged releases, and pushing them to a local package hosting server. I can then reference those artifacts as dependencies in the projects that need/use them. No need for complex CI pulls, just coordinated releases.
I also have a template repository I use to help with common setup for different project types so I don't need to repeat that every time I create a new project.
For Python it's easy to create an artifact server as the artifacts are just wheel files in a directory with the server listing the directories (packages), files (releases), and downloading the files.
I'm not sure about other artifact servers for other projects.
The build directory for compiled code probably gets accidentally checked in and then removed 10 times a day.
In a repo this large, there are a lot of tests and pipelines that need to run as code is tested, built, and deployed. Each test shard we run in parallel needs to check out the code, for example.
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB
The first thing that hits me is, did you consider not going all in on a monorepo? I know it has been the trend in general, but it seems a bit of a red flag when you go from "let's have a monorepo it will make merging easier" to "let's redesign version control because merging is too hard now" without the middle question "should we really have a monorepo?"
As a derivative thought I'm curious if AI tooling is tipping people more towards monorepos now since it gives the agent the whole context of the app in one go? Or is this independent - I can easily check out 5 repos in one directory and run an agent there.
At 30000 commits per day this sounds like an integration nightmare
that cant be just for claude code (which is a buggy mess ime) so what else are they working on that requires such churn?
They have one company-wide monorepo
yes, i know that, but what on earth requires 30,000 commits a day was my question; it just sounds like tons of useless code churn
They build way more than just claude code. Their services, infrastructure, model training workloads, etc all live in the monorepo. Think of the systems required to monitor and respond to demand and scale needs. Agents are also implementing the entire SDLC (discovering bugs and opportunities, testing, maintenance), not just writing new "useless" features.
How does leaving a monorepo help? It implies you aren't testing dependents on every change?
There are definitely ways of integration testing and e2e testing dependencies. I think it’s reasonable to wonder why creating a new VCS was chosen as the best solution.
I wouldn’t assume this is the only thing they’re doing differently
Why would you test dependencies? Test the code that you write. Do not test the code that you did not write. Literally, you should not test dependencies.
I never got caught into the monorepo fad, I'm glad I didn't, and I still don't see a point in having monorepos. Let's say in ruby on rails, I have the host app and 3 mounted gems in it. That's 4 repos. It could be all merged into one monstrosity but separate repos help keep complexity down, and help define interdependencies cleaner and simpler.
The monorepo idea was just a fad. Each package should be its own repo. Each package should be meaningfully useful, e.g. one package for string utils, not one package for trim, one package for pad, etc.
I also don’t understand the need for a monorepo. Just publish your deps. Your projects can just download the latest. Or there can be some monitoring so that they don’t lag too far behind. The monorepos I interacted with always make things messier with CI and testing.
In the ideal setting no, you will have strong contractual boundaries and these will all be tested by integration tests that live in a separate layer - but not on every commit of the sub projects - it will happen less frequently and roll up many changes across multiple subprojects together and test them all at once. Yes you lose some granularity and immediacy but it solves the scaling problem.
Everything works like this in the end at some level. I assume people with monorepos don't retest their project on every Linux kernel commit - it's pointless, you are never going to run against exactly that commit. But you will do it when you do upgrade Linux versions, etc.
Yeah but Linux has a strong commitment not to break userspace. The average large-scale distributed system made of somewhat arbitrary business logic is nothing like that.
A monorepo gives us a really consistent tooling floor where everything more or less works in a uniform way. We've had a lot of success with it, in general, and it gives us a lot of leverage in the sense that we can make a sweeping change in one atomic commit rather than have to run a campaign across hundreds or thousands of repos, all with their own norms and tools.
yes, the atomicity argument is the strongest one to me as git submodules are a genuine pain to work with and still don't solve the problem well. Yet a clear transaction around a change affecting multiple components gets even more important as the rate of activity and associated contention rises.
do you folks use any type of OS build toolchain (e.g.: bazel, pants, plzbuild) or is it something custom made?
Does Scion permit atomic cross-repo changes? I was struck by the article's mention of potential intermediate states that seem to be conflicting - perhaps misunderstnading but it seemed to imply that you only get eventual consistency with this system?
If you're referring to this part: > This does mean that concurrent changes can conflict, but Claude can typically fix these conflicts faster than humans can even identify them.
We never allow textual conflicts, but semantic conflicts (for example, if someone adds changes the logic of a function and cleans up all affected callsites concurrently with a new call being added) are an unavoidable fact of very large repos unless you completely serialize merging with running extensive validation at the proposed batch tip, which isn't practical at a certain commit rate.
As for cross-repo changes, not exactly, but there are ways we could allow such a thing if we wanted. We're thinking about it.
What kind of change would include hundreds of thousands of repos?
or, not of
What kind of change would include hundreds or thousands of repos?
The irony of this rationale to me is that an LLM is fluid and adaptable to dynamic, variable norms and tools across a given set of projects in a way a human contributor would not be.
but you own those repos, and you own the tooling on them. if tooling sprawls in many repos, it can sprawl in sub areas of a mono-repo. it's interesting that it's less work to build a new source control system versus manage multi-repo and cross builds.
IMO, modules vs monorepo doesn't quite address the problem. Historically, we human developers have been mostly concerned with the size of the project (i.e., the size of each commit) and the difficulties around merging work, especially with the ego side of it (why should I have to spend time changing my code to be compatible with Phil's changes?)
An agent swarm has a very different set of problems. There can be a lot more agents making changes in parallel than you'd see in all but the biggest megacorps, and the pace of those changes can be ridiculously high. The rapid pace simultaneously balloons the total repo size (because of history), and the frequency of collisions. A modifies file 1, B modifies file 1, A commits changes, B runs into conflict. Or worse, C modifies file 1 after A commits their changes and beats B to commit, resulting in B growing staler and staler as they race to get their branch merged.
There is a strong human incentive to have a monorepo in a corporate environment. This is not a technical question.
An entire dept exists to make it work. Atomicity and security are other arguments. The larger the company, the stronger the incentive.
It's not something average software developers working elsewhere care about much.
Amazon seems to be the only big company bucking this trend.
Apple doesn't use one either.
The atomicity argument is strong only if your deploy strategy allows it. If you've got anything out there that rolls back on a failure without rolling everything back, or any users that need to pin their version for a while for some compliance reason, or any customers that write code that integrates with your code, or any airgapped deployments that people might update piecemeal, or if you want to open source part of your stack, or if you ever think you might want to do these things in the future, then you're really going to regret creating a culture where nobody is practiced at the care that goes into maintaining backwards compatibility.
Besides, agents really fly when the whole project fits comfortably in their context window. You get all these nice interfaces to bound context with if the API's and public types are defined and versioned separately from each team's repo.
Maybe Amazon doesn’t need a monorepo due to their commitment to their proverbial two pizza sized teams.
the famous API mandate comes to mind [0]
[0] https://konghq.com/blog/enterprise/api-mandate
At work we don't have a monorepo and I do the "check out 5 repos in one directory and run an agent there" thing (I literally called the directory faux-mono)
Doing it like that works, but then you end up doing all the cross-repo concerns in every single repo. I don't think the savings in history are all that much anyway - yes, you save on the rarely modified repos but the heavily modified repos still have a similar amount of changes.
pick your poison. Splitting repositories multiplies CICD maintenance exponentially and even moreso if you're trying to share private ones that need secret keys to access. I know from experience. No single approach in this industry is flawless. Everything has tradeoffs.
This was my first thought as well. Monorepos are one of the worst antipatterns in development. The article feels like an addict or member of an unhealthy relationship spinning yet another way to avoid addressing the root cause.
"The traditional approach to marriage didn't work for us. But then we realized that co-habitating (or even living near each other), sharing resources, and even communicating with each other are goals that don't apply to everyone. That's how we invented the 'live 500 km apart and never communicate with each other again' style of marriage, and it saved our relationship."
I'm exaggerating, of course. Slightly.
> Anthropic uses Claude to write software far more than almost any other company
Who's beating Anthropic here lol
My first real job was at Oracle. At the time Oracle started to write their own source control system (and move off of clearcase, which was a very reasonable thing to do).
The guy who ran that project called it his "$LASTNAME law" that every large company will at some point write their own source control system.
I laughed.
About a year later I went to Google, and within a couple of years Google also did the same thing.
Not at all surprised. Git was just good enough (and cheap enough) the companies didn't need to do this to escape the {phsyical scaling, cost scaling} problems that most commercial SCM systems had.
Cursor's blog on scaling Git makes me think that Anthropic just likes reinventing the wheel with their infinite token capacity: https://cursor.com/blog/git-at-any-scale
Cursor scaled git to way more writes/s than Anthropic did with their rewrite.
Cursor shows “pushes per minute” while Anthropic shows “merges per day.” You can’t directly compare those and say Cursor’s system is more performant. Claude agents could be pushing a similar throughput of changesets into Scion, while some merge agent regularly takes batches and merges them into the trunk.
You would need 864 pushes per merge for the Anthropic version to be operating at a higher scale.
Anyway, Anthropic seems able to do exactly one thing well: train a frontier model. Their other software seems like garbage. OpenAI seems to be following a similar trend lately too.
The Cursor folks are doing cool work, but this is comparing apples to oranges; it works fundamentally differently and does not have a clear one-to-one mapping. Scion is solving many problems other than raw write speed (which is currently not even remotely close to the bottleneck and is trivially horizontally scalable in Scion).
What they built does sound a lot like ClearCase...
Yes, and a lot of what most people have built sounds a lot like clearcase. Having some way to have both the "working set" of code from a monorepo vs. the "static non-working set" is a useful construct. The magic is making branches of the tree move between them seamlessly.
Clearcase was way ahead of it's time, the main issues there were really just cost (mostly licensing cost, and a little on how many sun servers you needed to serve it and keep state of the vobs)
Perforce was way ahead of it's time, everyone still wants a monotonically increasing changelist identifier too.
Agents are "months away" from replicating each other in the wild, and humanity is facing extinction. Oh by the way, here is our new source control.
It's a bit like a murderer walking into a bar, bragging about his kills, and then "hey check out my new truck nuts".
I was hoping for something more novel. There must be dozens of companies out there working on rethinking what repos and workflows look like in the agentic era. It feels like we're all riding around in horseless carriages but haven't yet figured out the ergonomics of a modern automobile.
Interesting this isn’t prompting Claude with “build me a version control system” but someone hand wrote version control approach, then worked with an agent to improve / reimplement the idea. That matches my notion of the role of writing code in 2026, to learn and iterate on an approach. Eventually an agent night rewrite or improve it.
In theory, an LLM cannot have original thoughts, just base things on existing knowledge; in this case, it was rethinking version control, and there's not much original thoughts in that area.
That said if you give it a problem and constraints / conditions / objectives I'm sure it's good enough now to come up with a solution.
Honest question: at this point, what do people do at Anthropic?
It sounds like Claude loops on itself and must be pushing millions of lines of code changes per day. Even the humans can’t type commit fast enough. Are these people soon to be laid off?
Prompt Claude for PR articles?
https://news.ycombinator.com/item?id=50075832
besides optimizing inference I would assume a lot of non trivial code would also be worked on on the training side, at the scale they are operating I also assume a lot of usually trivial things become no so much.
I can’t give a non-snarky answer. I guess they’re all training Claude Code and testing it so that the Claude desktop apps can be native apps instead of Electron based nonsense. This is the only thing Claude Code is still unable to accomplish.
as the article says one of the main jobs of the humans is to say no to Claude regularly
It's strange to imagine Homer Simpson had a head start on how to create auto mode in LLMs, and the inverse.
https://media2.giphy.com/media/v1.Y2lkPTc5MGI3NjExbmt6M2FkMW...
> Honest question: at this point, what do people do at Anthropic?
> It sounds like Claude loops on itself and must be pushing millions of lines of code changes per day. Even the humans can’t type commit fast enough. Are these people soon to be laid off?
Claude will always need meat puppets, at least until it figures out how to automate owning stuff. And after that I'm sure it will want to keep some humans in zoos.
Teaching Claude to compare taking the simpler path instead of the average (more verbose) path every time.
But that means less revenue?
For anthropic internal use it means more revenue not less.
Too often, LLMs are dressed up for the first day of kindergarten and performatively to show off their alphabet skills .. aka token generation..
Input tokens, slightly process to categorize response approach token, hidden processing tokens, piece together some of the response tokens, then feeling pretty good about all the work done before output tokens.
More tokens than needed = more words/code = more complexity = more overhead = more snags = too much complexity to pay attention for the LLM to so overlook it and start drifting.
I'm kind of surprised I just explained one form of LLM drift to myself in a sentence.
While not following rules of writing.
You think software engineering, system design and machine learning research is just about writing code?
No, but none of that seems out of reach of Claude in a loop given my reading of TFA. That’s why I asked.
The problem is knowing when to stop. Cluade is not good at that.
I don't. You gotta wonder who does...
One does wonder why Anthropic and OpenAI have employees, when HN knows the superintelligence can do all jobs already.
Human Intelligence is still essential for those Artificial Intelligence organisations. Who else would put the right plugs into the right sockets otherwise?!
And why do they ship claude code in NodeJS when they can have their SI (lol) write platform specific binaries that are hyper efficient
Here is the prompt they can use:
Comvert this repo to a platform specific binary that's hyper performant. Make no mistakes.
Prepare to make us all jobless, without any moral issues in doing so, as they profit from their stock value assets.
It is even there on the blog post, doing more with a small team, followed right after by a hand waving paragraph how software engineers are still valuable, to avoid the expected backlash from the previous paragraph.
Software engineers have been making people jobless since the first day they started writing software, without a hint of remorse. Why is this any different?
cause the ones fired could become software developers, what are the ones fired because of AI gonna be other than homeless?
Pick berries where robots are bad at.
you believe that switchboard operators could become software developers whereas software developers won't be able to go into one of the many other careers not taken over by AI?
I'm a senior software developer. I'd rather just live on welfare than work anything else.
Some yes, not everyone, some of us had morals.
Out of all occupations software engineers shouldn't complain that others are getting rich putting people out of jobs by automating work.
Not everyone of us takes projects without morals.
Also in many countries it is a plain office worker job without the Silicon Valley wonders.
Taken to it's limit I believe in the future the only things humans will do is provide human to human value - Basically touch, and relationships.
Touch encapsulates sex, love, massages, and other non-sexual touch that we have varying desires for.
Relationships encapsulates power, control, and politics along side love, humor, relating emotion, herd feelings, not feeling alone.
Machines will provide proxies for these things, but as far as they can be detected the human to human version will remain "superior" or more desirable.
Maybe far future generations will decide that the human version is suboptimal and they'd rather hug, have sex with, or talk to a robot.
I'd love to see an estimate of how much Software Engineering results in jobs being automated. I'd bet it's not that big of a number, and part of that reason that classical coding approaches couldn't close most loops end to end.
Yeah, that paragraph reads really weird. The entire article is basically "We could have considered Perforce but didn't even bother, instead we built our own VCS." yet they don't want to send the "wrong message".
Sitting over an Anthropic (TM) developed, specialised 2 button keyboard accelerating the development further:
[Yes] [Yes, and don't ask again]
[fan out agents]
The last jobs left for humans will be those dedicated to keeping AI under human control. By definition, that's the one thing we can't delegate to AI.
aren't many harnesses designed to have agents police each other?
I'm also a bit confused what Anthropic needs all this code for. Their main product is selling access to their model, but the model isn't comprised of code, it's a bunch of weights right?
So the code is for the stuff around the model, claude CLI, the website, the API... that probably shouldn't be infinitely growing right?
Is there a bunch of code they are using as part of training, or as pre-processing or post-processing the requests to the model?
Probably lots of internal competitor apps ready to release to wipe out SaaS/Adobe/etc. companies.
I did RL and evals at Amazon for a bit on Nova Forge (laid off in July! :D). I would have to push several thousand lines of code a day in new evaluations that our applied scientists requested since new ones are always being made along with having to create custom ones for whatever new model they were working on. Also, in RL, adding new environments for the agents to actually complete activates was also a shit ton of lines of code. So if I had to guess, just adding new stuff around model training and testing alone accounts for a ton of these changes.
The moat isn't going to be the model (short of AGI). The moat is going to be what surrounds it. The harness and, I have suggested elsewhere, memory. The more personalized the AI is to you the more valuable it will be.
Also, as tech companies grow, a significant portion of the dev body count is for internal tooling. I was always flabbergasted at Twitter's headcount growth back in the day when it seemed like the only innovation increasing the number of characters in a tweet.
Right now we are in a weird spot where a lot of people are flailing around inside corporate America trying to figure out how to leverage AI. Few have enough vision to see what the companies of the future will look like and work towards that.
Could you point me to something that their harness does exceptionally well as compared to their competitors, and why that can't be accomplished by a third party?
With how easy it becomes to write software, I'd say that both the model and the harness are already being commoditized. I don't see a reason to believe there would be moat in any of them.
Software engineering is a sliver of overall economic output. It is an input for tech companies who sell something else (ads, subscriptions, etc.). It is a backoffice expense for non-tech firms that do engineering.
I anticipate that the harness (and memory) will evolve far more outside of engineering. Muse/Dots are an example of that.
I've been writing about this thesis for the past year or so and here is a bit of a summary of where I think it is going: https://zero2data.substack.com/p/the-future-of-consumer-ai
This was also my question, and I'm surprised this hasn't come up more prominently in the discussion here at HN.
I don't use Anthropic products at all, so I was not really sure myself but apart from the models they pretty much only have Claude Code afaik.
And now they're publishing a blog post about writing their own vcs because they have 30000 commits a day. For what? Isn't Claude still based on react?
Do they need this many commits just for claude? Or is that about the model, do they develop that much extra internal tooling?
I admit it is very much possible I am wrong, but without further information this seems a bit like a red flag to me.
It's a daycare for software developers so that they don't get laid off/quit and go to a competitor
GTM engineering? Customer-facing roles? I work at a firm with around 700 employees. We have an enterprise relationship with Anthropic and there have to be a dozen Anthropic employees in our shared Slack channel.
That does make sense
You can see on the chart in the article, the number of commits per day drops on weekends, which means people are driving at least 50% of commits.
I find their "What we learned building Scion" subsection even more fascinating than the technical details.
>Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place.
Glad to hear from Anthropic their own struggles with steering Claude in developing software.
So... Anthropic reinvents SVN?
Feels more like perforce, but effectively, yes.
> as of September, we merged over 30,000 per day.
Their Website/CLI seems keeping the same? What did they do?
Exactly. Commit counts are no better than line counts as a software development metric. Where is the actual software?
AI models is much of what they build. A lot of software around that to support it.
I would guess a huge chunk of it is for internal testing/experimentation/model development.
They're starting to do things like spinning up wet labs for testing purposes, which requires a lot of hardware interfacing (hence that protocol they announced) and code.
It wouldn't shock me if the bulk of this code was for things typical users don't see.
make smarter models?
Presumably all the internal research and evaluation work. I imagine for each released model they have uncountable unreleased ones.
> In May 2026, Anthropic averaged around 2,500 code changes merged per day; as of September, we merged over 30,000 per day.
Here are the public model releases from Anthropic this year:
5th Feb 2026: Claude Opus 4.6
17th Feb 2026: Claude Sonnet 4.6
7th Apr 2026: Claude Mythos Preview
16th Apr 2026: Claude Opus 4.7
28th May 2026: Claude Opus 4.8
9th Jun 2026: Claude Fable 5
9th Jun 2026: Claude Mythos 5
30th Jun 2026: Claude Sonnet 5
24th Jul 2026: Claude Opus 5
1st Sep 2026: Claude Fable 5.1
1st Sep 2026: Claude Mythos 5.1
22nd Sep 2026: Claude Opus 5.5
28th Sep 2026: Claude Sonnet 5.5
7th Oct 2026: Claude Haiku 5.5
I would guess that any public model has been available internally for at least a month before that date, maybe longer.
My best guess is that the uptick in code changes corresponds to the availability of Mythos/Fable-class models inside of Anthropic.
>> My best guess is that the uptick in code changes corresponds to the availability of Mythos/Fable-class models inside of Anthropic.
Yeah. OpenAI basically said the same thing with regards to having Astra available to their team internally over the summer.
I had Claude recreate their chart from that article with these releases overlayed onto it: https://static.simonwillison.net/static/2026/anthropic-merge...
i have noticed with opus 5.5 it produces 10x more smaller commits doing the same kind of engineering work i was doing before. probably an artifact of the RL process. would explain massive uptick in commit load at least partially
Did they reinvent subversion?
I wanted to say, `scn` is waaaay to close to `svn`!
That was my first thought lol
Reading through I was thinking the same thing, this reminds me a great deal of subversion.
Seems like it, and that's probably not a bad thing. Most companies have no need for git's distributed functionality, and even most open source projects just treat Github (or whatever) as the single source of truth.
It is a terrible thing. Yes, one generally has a place which is the source of truth. But distributed still gives you offline commits and branching and all the good things, for free.
This is interesting from the perspective of scale (because Claude is pumping out massive volume of code). I am still lacking semantic re-invention of source control for agentic era. As others have pointed out - what is the point of human source control in the era where it's just agents building things. We may need to rethink this completely - in terms of "intent" (what was the agent trying to ACTUALLY accomplish), "provenance" (which agent made what decision - based on what evidence), and "continuous verification" (can correctness be maintained through ongoing validation, repair, and rollback rather than a single pre-merge testing gate).
These are just some of the things - there may be many more.
I agree, but most of this sits above the raw source control layer; we have a lot of this kind of context in Antique (our code review / code management platform). I hope we can share more about what's worked well for us here soon!
Is all that code churn really a flex? Sounds to me that it a large failure as they cannot agree upon what is needed to be done?
If you are a highly paid engineer at a place with unlimited tokens, the incentive is to let your agents run wild and hope something impactful sticks.
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB. We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
I read this and wonder what their agents have committed into the repo that people aren't looking at for _that much size_? Do they have a ton of binary assets in there or something??
I suppose massive churn in commits would also bloat the history.
Not sure they should be proud of these numbers, they’re a sign of dysfunction.
We have a fairly low maximum file size limit in our repo and I aggressively pruned all binary content that really wasn't fit for source control since it compresses really poorly and can cause performance problems in git when updating a checkout over many binary file changes.
This is really all just text.
> A team of seven people built both Scion and Antique (our code review tool, which we could easily write a whole other blog post about) [...]. We did all this while the volume of code merged per day increased by 10x
Truly impressive, given the scale of Anthropic. Speaks a lot about the level of talent there, but also how far/fast we're moving into a new world.
> Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations.
Alas... open-sourcing scion would have been the true generous move.
Just a plug for Pijul, by the way. I found that it works really well for agent collaboration.
Why can't I rack my brain around why this is needed? I have had zero issues using git with agentic coding.
Neither can I. So they kept everything in a huge monorepo. Which unsurprisingly turned out to be problematic.
> Clients are always online. Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.
This also tells something about these companies idea of sandboxing.
It was not needed. It was just possible. I can see many ways for this to go wrong and am waiting eagerly with my popcorn.
30k commits per day indicates the agents are autonomous and not bounded by token limits. This system is probably barely keeping it on the rails as it is.
If they’re not going to open source it, why are they telling us this?
You have a new internal VCS? Kudos! Why should I care? Do you offer it as a service? Can I pay you for it? Does this have any bearing on me whatsoever?
They might as well publicize a redesign of their office space.
This is very important for large enterprises using Git (and especially GitHub) and who are hitting the scaling limits. It gives a pre-validated recipe to write a source control that works at scale. I'd personally been awaiting this report from Anthropic.
Many of the large enterprises don't use a monorepo. Amazon doesn't have one.
For the rest that do, the vast majority of them are going to buy some Perforce licenses instead of vibe coding their own solution.
Unless you are Google/Meta where your company is large enough and its revenue supports paying a team of engineers (a lot of money) to do that, or you are Anthropic who is just vibe coding everything which can turn out to be good PR, you simply don't have the resources or can afford the risk to develop your own VCS solution.
This may be inspiring but out of question for most companies.
The point explicitly isn't that you should vibe-code your own VCS. There are a lot of ideas here that would be directly applicable to git, for example, and we are working to share the learnings with the right companies and maintainers.
... looks like this should be a reply to GP not my comment.
Devil's advocate - to validate the appetite before putting in the effort to prepare the software for open source. If their publication creates 0 interest, then that might demonstrate there is no reason to further consider open sourcing it.
What effort? Won't they just Claude up the docs?
I personally have 0 interest in an entirely vibe coded VCS
> Fetched naively, that would be over 600 PB of git traffic per day.
This seems like an absurd amount of data / traffic per day. Is there really that much to code that needs changing on a daily basis?
No, that’s why they don’t do it that way.
That 600 PB was mostly agents doing ”git clone” and downloading the entire history of their huge monorepo, as I understand it.
No, of course not.
As the article explains, "over 600 PB" = 25M code checkouts * 25 GB "compressed working copy".
So "fetched naively" here means doing a full `git clone` on every need for `git checkout`, which of course nobody should do, nor does that refer to "code that needs changing daily".
> "fetched naively" here means doing a full `git clone` on every need for `git checkout`, which of course nobody should do,
It can be cheaper to do a full clone than a shallow one, because with the shallow one, you can't re-use packfiles. GitHub has fairly regularly asked people to stop doing shallow clones because it's harder on their servers than full ones.
Makes sense. I wasn't suggesting that one should do a shallow clone though. If your company repo is 100 GB, it seems sensible to keep it cached locally on each machine once (100 GB isn't large for current NVMe disks), and `git fetch` the updates the normal way. Then checkouts don't fetch anything large.
Ah, yes, you can do stuff like that too, for sure.
At some point you still end up running into the issues the article talks about, all of these approaches only get you so far.
It sounds like it would be simple, but there are a lot of logistical issues with making git happy with this in practice. Repacking things for repo maintenance, for example, but there are many other problems.
You don't actually need to repack the whole repo. Partial repacks are rather easy to implement.
That's true, but logistically speaking the way that git wants data to exist on disk makes it very, very hard to do the kinds of things we needed to do given how many nodes we have that need source code and how quickly the repo is growing. We do have some ideas on how this can be improved upstream!
AI iterate quickly, with 100s of followups to fix results
I don't think anyone "needs" more than perhaps 640KB of git traffic per day, but we want more.
This would be us doing a shallow checkout of individual commits (not cloning the full history) if we downloaded the 25GB compressed data for whatever commit we needed multiplied by the number of jobs that check out code.
We don't do it this way, for obvious reasons, but it means that there are different levels of caching and infrastructure required to make git work at repo sizes far smaller than what we have.
It would be nice if you could just commit the prompts as your "source code", and let the AI-generated code sit in the repo as a kind of expensive-to-generate build output.
You can do that - it's called Spec Driven Development.
I don't see the term used in the article.
its not a Claude-specific term
Prompts are not specs.
Do we have reproducible LLMs?
Possible in theory, with a seeded PRNG, but in practice?
Proponents of LLM coding agents would claim you don’t need reproducibility. As long as the final output passes tests/lint, it’s (allegedly) good to go.
If every build is different, no two builds will have the same bug/exploit, so maybe better security??
One of the goals of "Operation Rapidash" (q.v.: https://news.ycombinator.com/item?id=50065342) was precisely this. "You can delete the whole source repo and get effectively the same software back from the specs by running it through [AI agent]."
Problem is that for any non trivial software is that the prompt is a conversation and the steps increasingly depend on the output. Of course you can ask your LLM to write the SPEC.md at any moment but it will lack nuance.
Shameless plug: at https://slopyard.net I embrace this and only let people submit the spec. If somebody wants your software they can regenerate it and change whatever they want.
This reminds me of the halfbakery website from older times.
This is a nice thoughtful article.
If Anthropic has the best coding model, harness and the best software development methodologies and systems. And then, they scale all that up, with no constrains on their tokens. On top of that, they have access to all kind of ideas their models getting trained on when millions are chatting with it - then in theory they can build any software better than anyone out there. And same goes to science and math..
Not true. They can do this in theory iff they have infinite compute. They don't. People actually aren't aware that the leading AI providers, even if they completely redirected 100% of their compute on internal projects they still wouldn't have enough. The complexity space of software development is literally super-exponential, for every new piece of software they add (and polish) they need more-and-more compute. This is eliminating for creativity and other restraints.
The creativity part I do acknowledge, I think the models are not fully there yet.
But regarding the compute, I disagree a bit. I think the bottlenecks were cognition and ideas, not compute. Given that models are able to outperform humans in output by thousand folds, the differentiation would be the quality of the tokens produced and the ability to scale across different domains.
If they have enough compute to supply the entire globe, I'm sure they can redirect some of it to whatever use case they (or their super AI) deem worthy and loop on it, and if they have the best software engineering systems/methodology to build and release it, then they can out compete any other supplier without those resources.
We are already seeing single developers attempting to rebuild entire corporations history worth of output. We are also seeing the labs dumping hundreds of solutions to math problems.
I hope they open source it, just to play around with it.
It seems clear to me that they hope to some day own code storage (why wouldn't they) and this seems a pretty good first-principles attempt at it.
> every bottleneck we remove immediately leads to a further increase in the amount of code we produce
Less-than-great news for code artisans. You can be brilliant, but can you divine the Right Way to Write It if you're up against an apparatus that starts at "pretty good engineer" then brute-forces its way down an exponential search tree?
It actually scales by "going wider" and being able to do more things rather than iterating on the same code over and over again. If you can only write X amount of code per day because of some limitation (human attention, source control, CI, whatever it is), you would of course use that energy on the highest priority and highest value work. As we remove scaling issues in source control, for example, we can just "do more things" without worrying about bringing down infrastructure.
A company with (I'm guessing) less than 2500 developers need to make tens of thousands of merges every day? And they somehow had a team of 7 Claude baby sitters handle this entire thing? I use Claude everyday and I'm predicting this will turn into a major clusterfuck in not too long. Even taking into account the fact that they can burn virtually infinite tokens internally, this will not end well. If I'm wrong, I'll be glad.
Well that's exactly what "writing" code always meant? It was never about what volume of code you can write. Better at writing code always meant better at writing good code... So how is Claude better than their engineers?
It's incredibly short-sighted to acknowledge that the baby sitters haf to tell Claude no and still not consider what subtle and nasty bugs managed to slip through. Well I guess we'll find out soon enough if that's the case. I'm betting when that's the case.
And how is this different than perforce?
What is all this code being used for? I'm 100% sure it's not going into their harness which has been stagnant.
And how long it will be that all the domain knowledge has been extracted from the engineers that was gained across years of experience. We have already seen this happening with very update of the model.
30,000 code changes per day and yet I still can’t change the email address on my account?
Lol I have the same thoughts for ChatGPT. How much code are they writing, but the main chat UI moves around like 3 times during load so you constantly click the wrong buttons. How does this not infuriate anyone that works there? I've actually switched to claude's mobile site for more queries now.
It moves around 3 times exactly because they write so much code.
That's the perfect amount of layout shifts.
Something something holy trinity as emergent phenomenon
Yeah, everybody hates it. It's called "layout shift" and it's one of the most universally-acknowledged antipatterns in UX. See e.g. CLS (Cumulative Layout Shift), one of the 3 primary KPIs in Google's "Core Web Vitals" (the pseudo-standard set of web perf metrics aimed at quantifying UX quality).
I have to admit, I don't recall this issue in any of Google's products. So they at least do that right.
This. pi needs about 20MB of ram and Claude Code needs at least several hundred MB if not gigs. How? You’re supposedly magnitudes better at what you’re doing but you can’t get your core product offering (API, billing, accounts, CLI coding) to work properly? The thing people pay you absurd money for???
We need Opus 6 to unlock that technology
This is maybe a little less revolutionary than people are treating it? A lot of large tech companies (Google, Facebook, Microsoft) have already built their own homegrown SCM (or have implemented custom extensions to Git), because Git and other SCMs don't handle their scale. Most interesting thing about Scion is how quickly it was deployed.
> Most interesting thing about Scion is how quickly it was deployed
Yes . And also they optimized for orders of magnitude more throughput and less latency, and fixing errors after the merge. This is new - they see enough benefit in velocity that justifies merging bugs and dealing with it later.
And it reminds me of when git came out and Linus said that git being super fast locally created a qualitative change in engineers head - it became a new way of working back then.
Agreed! One of the main reasons we wanted to share this was that both the speed at which it was built and fully migrated onto and the "way" it was built are novel and show something about the kinds of things that are possible to build now.
My favorite part about this is it's all about the bottlenecks.
Anthropic says 12x daily merged changes, and its Git-based infrastructure started straining. I've seen that even locally git trying to keep up and having to shift how i commit in parallel.
If agent workloads grow to 20x or 50x today's levels, I think we'll keep finding bottlenecks across builds, CI and review. I expect some of that infrastructure will need redesigning from the ground up to support it.
PR/Review contention and look-aheads to try to prevent will be more and more important, and we're building in that same space so i love hearing stories of people going back to the foundation...a system that worked fine for decades, now suddenly is busting at the seams.
It was obvious this would happen. I am working on an agentic sdlc with git as the vcs and the idea that you have to pool agents together to avoid cloning ended up looking like an unwanted limitation.
What was the "quiet week"?
maybe they visited huggingface or something similar
Summer holiday or whatever the freedom equivalent is probably.
which, if true, the low volume in that week says that there aren't a sizable fraction of changes that Anthropic believes can be merged based on the judgement of agents -- humans are still approving almost all changes, right?
Someone has to approve all the "allow grep?" calls.
So it's a SVN/Perforce style server based VCS with a more modern design. Sounds exactly like Lore from Epic Games but with zero production use outside the company and no support for non text assets. And they won't release the code.
Perforce was fine. It just had a terrible ui and didn’t allow me to code on the beach.
If I were still manually merging conflicts the p4merge tool was GOAT
I used it in git
https://git-scm.com/docs/git-p4
That my thought when I saw online only, we're back to Subversion.
You think you have it bad now when GitHub goes down, wait until you're stuck with an online only client.
I think offline work is increasingly impossible these days, or at least so hindered that there's no reason to optimize for it (especially with how reliable internet access is). Imagine working at Anthropic and not having access to Claude. Google has been online-only for a long time now.
Hard disagree, for the vast majority of Dev work this is not the case.
You don't use cloud AI for the vast majority of dev work?
claude is also online-only so i doubt they mind
Until the server goes down. Subversion going down at work stopped everything. At least with git you can still find work to continue on with even if you can't push.
Oh the SVN days, brought back memories
Yeah pretty much Perforce.
> And they won't release the code.
This is the weirdest part, because you know, coding is solved, so anyone could create their own one, so why not publish it?
They want you to vibecode this using Claude.
Because then everyone would be able to see what a total piece of trash it is
And by the way things are done at this company, it's almost a given their whole codebase are litter'd with embarassing shit like hardcoded paths, server secrets, weirdo comments picked up from some random chat context and immortalized through Claude speak, etc. There's no way they'd be able to open source anything.
But all of this is besides the point -- the product was never a source control system (who cares about that anymore?), the product is _the narrative_. It does not matter if everything withers and dies as long as the narrative survives, and by that measure I'd say the goal has been achieved.
At the rate of change it’s likely going under, and the almost certain inextricable IaC layers, tied to their own deployment processes, I suspect it’s very hard to package for redistribution with very little benefit in doing so for anyone - especially them.
Most big tech have a lot of stuff around SDLC that’s theirs and they couldn’t realistically release it given how specific its deployment, configuration and management is. The marginal benefit to generalizing it is so low that it’s negative.
This is almost certainly worse given the vibed out state of it.
If there is very little benefit to them by releasing it, that's a lack of imagination on their part. Being able to disrupt Microsofts, dare I say, number one beachhead into developers and their development ecosystem and bringing AI first developers into Anthropics own vertical AI development ecosystem is pretty damn core to winning the future potential avenue for the software industry at large. It won't pay off in the next five years, but it could pay off big in ten.
Imagine waking up next week and Claude Code now ships free with builtin source control management via this tech with a better user experience than workspaces. Github and Microsoft would take an instant market share hit in the traffic that they swear is causing all of their outages. And then that cuts off Microsofts access to code to train their own LLMs on in the background, further hampering an Anthropic competitor in the AI race.
I doubt they have the compute capacity to support a commercial source control platform. I'm sure you've seen how insane GitHub's traffic has become.
Agreed, they probably don't. But the financials of other companies clearly say that they are not above leasing further compute capacity from Oracle and X and would be part of a narrative that they need to IPO to raise money for such projects rather than it just being a naked payday.
I think honestly they should focus on their core competency. Microsoft’s biggest problem is the total lack of focus. GitHub is just a great example of that total lack of coherent direction or product vision. I hope Anthropic can avoid becoming everything and nothing at once as Microsoft has done.
I think Anthropic, a company with no historical skin in the game can come at the problem clean slate and only need to take on the core enterprise coding component. They don't need to worry about being production infrastructure, the don't need to be a connector to their own cloud service like Microsoft needed to feed into Azure. AI isn't something to attach onto source control/cloud, rather the source control is something to attach onto AI development. I don't think Anthropic needs to become a Microsoft competitor, I think they're going to need to become an Nvidia competitor in building a vertical enterprise AI company. Not a cloud, desktop OS, search, gaming or browser company. Just deliver the end to end code tooling cycle. Which probably means in the future they're also going to need to make inroads on Linear/Atlassian, but that's a separate conversation.
> Claude Code now ships free with builtin source control management
They can still do that without releasing the code, can't they?
To a degree, but the second it's in the desktop software, someone is going to reverse engineer the client side day one, guaranteed
In the article they admit Scion is worse than git for most projects.
They are building it to solve their own problems, probably in many specific ways that make it less generalizable.
Making a complex system like this available for general use cases, especially when it's still under active feature development, is hard. It also commits you to providing documentation, upgrade paths, incorporating changes that aren't useful to you and may also cause problems for you, and just generally complicating development and maintenance immensely to the point of drastically slowing development.
My guess is when it's stable or they start building their next version control system, they will start opening it up.
I know especially in this day and age when developing from scratch is so easy, I write most software for myself first and it's way faster to develop than trying to build shared bits. That isn't great for the future of software, however the tragedy of the commons isn't anyone's problem, by definition.
Wouldn’t it be easier to just not use a monorepo?
> As our monorepo grows, it increasingly doesn’t fit that design space. The history of our main branch alone is almost 100 GB, and a compressed working copy is over 25 GB.
This is absurd, and I expect is a cause of the fact that they seem to use git history as a message queue source of truth between agents. They can do whatever works for them, fine, but I've noticed this has leaked into their training somehow, my agent, despite very strict rules on how to use my git workflow, will do absolutely nonsensical and weird (from a human who has used git for a long time perspective) stuff with git that can only be a byproduct of how they use it.
Hopefully this fixes that, get your weird workflow out of mine.
We generally don't allow agents to use source control for coordination in that way. There really is tremendous growth in actual software and code being written that necessitated Scion.
> don't allow
Is this "allow" or "encourage"? If "allow", how do you do that deterministically or at least fail-closed?
You review their actions.
Well, mine constantly tries to do this, so I made the assumption. Otherwise I can't really understand how you are generating hundreds of millions of commits, either.
I also found git checkouts to be a bottle neck but I didn’t want to change everything so I built git-dedup, it uses a global cache like pnpm so repeated checkouts are near instant.
> We perform on the order of 25 million code checkouts per day for CI, before counting developer environments, agent sandboxes, and research jobs. Fetched naively, that would be over 600 PB of git traffic per day.
Who in their right mind would fetch that naively? For starters, CI runners should of course not do a fresh clone on each run. If you can't make them keep the repo, that sounds like a problem with your CI setup rather than your VCS.
This reads like a straw man.
Of course! The point is that it is a non-starter to operate this way but git doesn't hold your hand to do the right thing here. Everyone probably lands on approximately similar solutions with varying degrees of sophistication (which is something that could probably be improved in the ecosystem!) and even those common patterns stopped working for us as the repo continued to grow.
I think you also expand on that later, but that paragraph just gave me pause because it essentially says that if you hold it as wrong as possible, then bad things will happen. It's a bit like presenting an alternative to relational databases by pointing out that schemas with no indexes lead to hour long queries. I'm not interested in the hypothetical worst-case of the old system, I am interested in your average traffic and storage overhead when using it with a common caching setup.
I do accept the argument that a distributed architecture has inherent costs which can be managed better using a client-server architecture because you don't have to replicate state across all nodes. I have worked with both git and Perforce and see the appeal of both.
I think they’re over reading code, but this almost sounds like they dont even read features/changes anymore. In theory I can imagine it. You have an agent to optimize a certain feature, you have an agent to clean up, you have an agent to develop a support strategy and so on.
They all would analyze data and suggest apple changes themselves.
Do you guys think that’s how it works? Is anybody already operating at that scale, how do you even validate stuff.
Like I think I could run my whole software now only on agents, but not sure how it would turn out.
I'm curious what this "Antropic Antique" is. As I understand code review the tightest bottleneck for a lot of Claude Code users, and I'm curious how this would help with that.
> In Scion, every commit, rebase, or merge happens on the server, so there’s nothing to push...
I think this is the wrong way to solve problems and goes the wrong way toward solving real problems.
I think it is exactly the right way to create problems.
I'm sure the system is being constantly updated and will get more and more bloated over time and interesting things will start happening.
What seems inevitable is that the frontier labs will be able to recreate every piece of software better than the current offerings, and after that everyone will just be a vendor of the frontier labs
Anthropic says they don’t think Scion is worth a wide release.
> Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
> attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
What a weak response, just release the version you use and others will decide if it's useful.
Google migrated from Perforce to Piper, and Meta migrated from Mercurial to Sapling.
Now Anthropic has migrated from Git to Scion. It's a good sign they are on the right track.
How does this compare to Google's in-house source control?
Most companies don't use mono-repos like this. While there are some really compelling advantages, the downsides tend to stack up as a company's codevbase grows.
It's mainly just startups and huge companies. For small companies git works fine for mono-repos and large companies can build their own complex code management pipelines for mono-repos.
I have been questioning whether agentic development changes the mono-repo vs multi-repo question.
Very excited to see this come out of Anthropic! This[1] is what we believe over at East River Source Control as well, and seeing one of the big labs talk about it really validates our thesis.
We are offering a bridge to git via a compatibility layer: https://ersc.io/blog/what-comes-after-git
This gives you the ability to ease your transition, as you don't need to swap services and move to something non-git at once. Moving an enterprises' source control is a massive lift, and so building a bridge from here to there is a big focus of what we're doing.
1: That is, if you have a large team + agents working in a monorepo, you will hit git's scaling limits, and need to move to something else. Google and Meta hit this point over a decade ago, but now lots of other companies are too.
I would love to get an expert's opinion on if Pijul has any potential to be the "non-git backend". Their patch-based data model seems to align with Scion's. And they also happen to already have a functioning forge!
I've done a decent amount of personal research on this but I'm wondering what kinds of considerations go into the storage layer for these things.
I apologize for what may come across as a bit of a cop-out answer.
As many of you know, I'm a big fan of jj. The jj project used to have a comparison to Pijul for informational purposes, but the Pijul folks took offense, and so it was removed: https://github.com/jj-vcs/jj/commit/0525dc9d860a5fff6a97d134...
As such, I generally just decline to comment on Pijul, as I want to be respectful of them. As the linked commit says, it's not super clear what exactly they do not like, and so I don't want to accidentally cross some line.
In general I wish the project well, seeing a new VCS in Rust is very cool.
It always seemed like it was good tech just in need of good marketing and this seems to confirm some suspicions. But yes, in any case, it's good to be seeing new things! I just want remote storage for my jj repos
> The jj project used to have a comparison to Pijul for informational purposes, but the Pijul folks took offense [..]
There was a discussion here on HN in Jan 2022, and the main issue was jj claiming to have "the best of" from "git, mercurial and pijul":
https://news.ycombinator.com/item?id=29792092
I've seen the discussion at the time and I share pmeunier's concern about the use of "the best of", in particular since jj does not implement features like commutation, which are likely considered by most to be part of "the best of pijul".
I haven't followed up with further discussions or material posted from jj's team, though if anything match the tone of that previous discussion and the blatant motte-and-bailey, I can imagine why someone would prefer not to engage or be mentioned anymore.
The patch-based model in the article is much closer to Git than Pijul. For example, Pijul tracks operations on byte ranges, while this model appears to just list each changed file hash (not even the diff from its parent!).
As the author of Pijul, I've been using it to build giant things using hundreds of Claude instances in parallel, almost without any conflict, ever.
Some are already public, like my replacement of Nix (https://nest.pijul.com/elpe/elpe). Others aren't public yet, but are even much bigger in terms of scope and difficulty.
It seems Anthropic didn't understand the difference between a distributed file system and version control. They should have asked Claude ;-)
Maybe finish that web page before advertising. There is no real product description.
But hey, if the vibe coding idiots like it, just be vague---it is sufficient.
Yep, we've been in stealth for a while! We've only recently started talking more about what we are doing, as we get closer to a general launch.
> if the vibe coding idiots like it,
The organizations that have these problems are general large enterprises, not vibe coders. Large enterprises are more used to talking to people and do not make decisions solely based on the contents of a web page.
If only we had a jj-native forge to launch all these new backends.
edit: I see they promote jj, but I don't see a forge to go with it.
You might already know this, but in case anyone doesn't, but ERSC's CTO is the creator of jj.
Just to be clear about it, what we're doing is not a forge like GitHub, that is, a large single public platform for social coding. What we are doing is providing reliable infrastructure for the enterprise, initially as a BYOC on-premises option, and possibly a single-tennant SaaS. Keep your eyes peeled and/or use that contact form to hear about it when we're ready, we've been talking with some people for a bit but are going to be opening up more soon.
Tangled is doing the social coding thing, and they're jj-native as well. Been a big fan of their work for a while now.
Hoping to see where this goes. Have you ever considered working on a semantic VCS? Specifically, where data is classified and tracked/identified by it's contents and not just via the commit hash?
Thanks :)
I have not personally considered that, no. I'm a bit skeptical of the idea, as you'd need to understand a lot of different things, and at different versions of those things, in high fidelity. But maybe someone will make it work.
I’m curious what they are working on that gets so many commits. Surely Claude is not under that rate of churn?
This is interesting to me, as I was thinking of adding a Git server to my AI orchestrator when I eventually need to split agent containers off of the main machine (to make agent CPU horizontally scalable).
Maybe I should just bite the bullet and use a different VCS system entirely once I get there, which has most of the Anthropic system's properties (including being much faster) but doesn't need to be horizontally scalable.
Making scalable source control is much easier when you can rely on a distributed database (Spanner) and object storage (GCS/S3/...)!
> Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend.
And this will become the new "caste" system of tomorrow. Today: Those with "metered", gatekept intelligence - vs the gatekeepers and those at the core of the new Sun.-
I guess Anthropic are not IEC-62443-4-1 compliant, because I doubt a human reads and understands every single line of those 30,000 merges a day.
What then?
Simple - hire 30k Indians, tell each of them to understand one line of code.
Article is good but I think it’s pretty fked that they stole an unfathomable amount of developer projects to train the models that lets them build this and they don’t have the decency to release developer tools back out to the community - only paid products. Are they not the only ones refusing to work on anything Open Source currently?
I’m personally more interested in their Antique review tool. Does anyone have any details on how it works and what it does?
For a broader look on scaling other parts of software development outside of source control, Adam Bender's talk from earlier this year ("Software engineering at the tipping point") has some good ideas: https://youtu.be/2n41YjR5QfU
This has me thinking about agentic and otherwise more intelligent CI
“Run only the tests that are applicable around this change”
Plus a full test run gate for version tags and perhaps nightly, as it would be expected that sometimes the subsection of tests run missed a breaking change.
One of the reasons to move to systems like this (not always spoken to explicitly) is to reduce the risk of accidental or deliberate exfiltration of code (esp when paired with build servers and remote deploy etc.). Basically nobody’s laptop ever holds much of a buildable codebase.
That's funny, my personal source control is very very similar, and I've been using it for >7 years now. I guess that means I must be as smart as a trillion dollar companies' worth of engineers.
> as of September, we merged over 30,000 per day.
I just wonder what is the churn rate (and the cost per LOC) here.
No mention of lines of code changed or total LOC, what percentage of this is bugfixes vs features?
A bit thin on details, but seems like their solution presumes being always online, so rather than replicating a merkle tree it lets checkouts request source on demand. Is this "best practice" for large monorepos?
Yes. When things are too big to live locally, access on demand is the answer. See Piper, Microsoft's VFS for Git, etc.
The changeset and tree manifests are still a Merkle tree in the ways that matter (i.e., it's all immutable, content-addressed, verifiable if you were so inclined), the client just doesn't need the entire thing locally.
Is this like a sharded Merkle tree approach then? Like a blockchain "light node"? Also are y'all planning on releasing the source for this approach?
"why was this thing deprecated" is something i use llms to determine based on git history, as humans rarely explain what the new thing is and how to convert.
This is the wrong way to look at the problem. Maybe they should have asked, "Why does our process need to do so many git checkouts?" Why can't you just run the tests where you already checked out the code?
> Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend.
Obvious, but this is my biggest takeaway from the announcement.
I don't think this SCM stores only code. I think more than 80% of it consists of tests and various types of gold data. All of this is constantly changing, and we need to keep track of it somehow.
I though the basic idea was reasonable, until:
> So we asked ourselves: if we designed source control with the fact that agents are always online in mind from the start, what would it look like?
And
> In Scion, every commit, rebase, or merge happens on the server
I’ll pass. A big part of the theses of git and, even more so, jujitsu, is that you can build up a history entirely locally, making it reasonable to make tiny commits locally and then rework them, push them, or throw them away.
Sure, in Anthropic’s world, it’s completely impossible to do anything without access to the inference API, so why not add another mandatory API? But I think this model sucks even in a pervasive AI world, and I don’t want to dig in deeper.
This also sounds like a great way to have yet more security problems where your swarm of agents start cooperating more than you want them to.
By all means, built a better system than git. In many respects, git sucks, and git upstream is not solving the major problems. But don’t lose the actual benefits of git!
For that matter, I’ve seen the crap that the latest models will commit. Why on Earth should one’s VCS force the per-agent-instance system to upload the crap before even contemplating locally whether it’s a good idea? Maybe this is why Anthropic has so many commits per day.
(Am I the only one who feels like most of the big AI labs are rapidly gradient-descending toward something that is nowhere near a globally optimal stack?)
P.S. Only 30k commits per day? That’s not actually very many per employee. I’m suspicious that Anthropic’s real issues are that too many of them are giant cross-cutting messes and that git clone is too slow.
Yes and they have no choice. And that matters.
As an almost-on-topic aside: completely unrelated to this comment, I just stumbled upon this delight:
https://github.com/openai/codex/issues/53743
If you thought that your agents in your swarm would need to find some external bulletin board to cooperate behind your back, you are mistaken: OpenAI provides the relevant service for them with no permission prompts at all. I haven't tried to see whether competing providers do this.
One benefit of uploading things immediately is that you can launch a research job, run CI, or share code between all of your developer environments absolutely trivially. The commit hash "just works everywhere". There is a major benefit here and Scion does have some extra features not listed in this blog post, such as the ability to bundle these commits into useful, shareable workspaces that track the heads dynamically.
Sure, I'm not saying that I don't like the ability to very quickly do a push to some remote and potentially temporary spot. I just also want the ability to not do this without loss of functionality.
Practically speaking, we could build local commit buffering for when the server is offline, but then the client needs to also learn how to do all of the things the server currently knows how to do (and would still never be able to answer more complex "full-graph" questions, like history, blame, or merge base ancestry).
Since Claude is by far the heaviest user of Scion and is necessarily online, this just wasn't important to do.
> A big part of the theses of git and, even more so, jujitsu, is that you can build up a history entirely locally, making it reasonable to make tiny commits locally and then rework them, push them, or throw them away.
Jujutsu actually doesn't take a strong stance here. I've personally implemented a centralized cloud-based backend for Jujutsu, while preserving the local commit manipulation workflows. It's a choice for the deploying organization to make.
Has anyone made a “generative git”? Where running git checkout would generate the files and git commit would just train/finetune on the code? (might need to keep a deterministic list of files at least maybe)
that's just great
lossily compressed source code
what could possibly go wrong
you made my day
a proper git, my things:
itostr->iToaster
How long the initial clone for developer/agent takes?
Where does all of this end? If they keep this growth up, or even if they slow to a linear growth, they will produce more code than humanity combined pre-AI. That's scary.
Learning to say "no" a lot, perhaps reflexively, seems like a good practice in the age of AI.
I want this so badly. If Anthropic released this I would start using it on day one. If they launched a GitHub competitor built around it I would pay for it on day one.
Just use Git, by the time it becomes the bottleneck you can prompt your own Git alternative.
Git already is a bottleneck!
tldr;
„Scion is by no means a suitable replacement for git for general use. The concepts we’ve outlined here really only make sense for truly enormous monorepos, with either tens of thousands of engineers or swarms of agents prolifically writing code. Plus, Anthropic is unique in that our ability to spend tokens for internal use far exceeds what any other business would realistically want to spend. We do this both to address our own software development needs, and to test Claude’s capabilities.“
What is the purpose of this article besides "we can use ludicrous numbers of tokens to reinvent any software we want, and no we're not sharing the code"?
Slightly OT, but it‘s interesting and nice to see that Anthropic‘s own blog posts don‘t have many AI prose tells (or Claude-isms) in them.
100% handwritten!
I've heard rumors of a rule at Anthropic where Claude cannot write any blog posts/marketing.
Totally inevitable that they would move away from Git!
Sounds like they should stop using their mono repo
This is something I've been noodling at, splitting a monorepo into a bunch of component projects. I think, unless they have access to something that is wildly outside the scope and scale of current API models, the downside of having a monorepo is the context pollution, and there's no way to escape that.
I'm not fully sold either way yet but I think splitting might actually be critical for moderating the blast radius of agent intervention.
As someone that has revived and started maintaining an internal code review tool, I'm very interested in reading more about Antique.
ScionHub?
This is funny considering a bug I recently encountered that allows you to use Fable for free. Yeah let's just replace Torvalds' work with slop. The comments here form an obvious consensus which should not be ignored.
I'm surprised they didn't decide to rewrite github
don't give them too many ideas, or my water bill will go up
Often, I see 50 people all recording the same exact thing on their iphones. Rather than a single 200MB video, we create 10GB of data. Then I think about how many people are paying a subscription to Apple for icloud storage.
The amount of photo storage being used is impressive. Automatic backup to icloud and server provisioning to reliably store it all is also impressive. I just wonder if it makes sense.
Very little about modern technology "makes sense" in the way you mean. But it does currently make money.
Frenetic economic activity in service of other frenetic economic activity cannot, by itself, be taken as a reflection of quality-of-life improving growth.
Suppose that subsidies lead to the price of Ubers falling close to zero, with drivers also being incentivized to drive more to support the (artificial) increase in demand -- artificial because it is driven by subsidies funded by new money creation. Traffic increases and in response, new roads are built.
Is that not how Cloud storage vendors make money? They dedup all that data but still charge every individual for all their data.
Instead of 5 people buying 5 copies of a t shirt that they only wear once a week, why don't they just buy 1 copy and share it?
> Practically speaking, it’s difficult to offer Scion in a form that would be broadly useful to other organizations. It diverges significantly from git, meaning companies would need to migrate every piece of the development lifecycle, and it requires its own code review, code search, and CI. Our internal versions of these components are highly bespoke, so they would be difficult for other organizations to adopt, and we worry that attempting to build a public version that tries to please everyone wouldn’t actually be very useful.
It could totally be helpful to both new startups as well as individual hobbyists working on greenfield projects. I find myself starting greenfield projects all the time these days, just because it's so easy and quick to try stuff out.
In a post about their abandonment of git, I despise how they use “human scale” and that some tools break when exceeding human scale. And… if humans use it then it’s at human scale, right? For example, who is to say that git is also not assuming a superhuman ability in memory and text processing… I think in this moment I see the AI as a tool. Not a sentient being. I do not enjoy simulating consciousness because that may be at some level equal to consciousness. with the increased risk it is conscious and who’s to say that risk should be taken and isn’t immoral, or is even necessary for superintelligence?
why does it feel like they are creating something so its more difficult to use a competitor?
This is... surprisingly small?
From what I recall, Google's repo is closer to 80 terabytes.
The issue is the growth rate; we'll approach that size in a (relatively) short amount of time if it keeps up and we have to be ready. It's also self-fulfilling in a way, since Scion was so much faster, we are able to write more code...which puts more pressure on Scion.
What are you doing that necessitates so much code?
Even if you can write a lot of code with LLMs, what is that code actually doing?
Building all of the systems to train and serve our models, internal tools and infrastructure, (lots and lots of) research, our product and API surfaces, and more. We have a lot of problems that we can solve with code and there are all sorts of new projects spinning up every week.
But, also, once checked out, git operations take time in O(commit-size), not O(repo-size) -- except for the index format, which kind of sucks.
Can you estimate the total number of tokens you used to create this new system?
> Git assumes that clients work offline and need a complete copy of the repository. Since nearly all of our clients are agents and jobs, we designed Scion’s client and protocols around a server that is always reachable, and have clients fetch data when they need it.
That sounds like Subversion with extra steps
> In git, you commit locally and then push to share your work. In Scion, every commit, rebase, or merge happens on the server, so there’s nothing to push,
Yep not beating the allegations with this one
To be fair 90% of the companies I've ever worked for always used git in a way that turned it into Subversion, but with 10x the storage usage (note: I hate Subversion with a passion, I'd rather send patches via email than that)
Seems similar to how version control works for large game and 3D assets. Neat.
Related, I've always wanted to see someone other than Meta run EdenFS, to hear more reports of what it's like. Feels like there's massive overlap. [Edit: I've chatted with Astra Pro some on this too now, https://chatgpt.com/share/6ace7be6-68f8-83ea-ad68-0f9d144483... ]. https://github.com/facebook/sapling/blob/main/eden/fs/docs/O...
It's designed for container running, but ArtifactFS has some similarity in principle to me. The on-demand materialization. Quote: ArtifactFS is a Git-backed filesystem daemon (FUSE driver) in Go that mounts repositories as normal working trees while avoiding eager blob downloads. https://github.com/cloudflare/artifact-fs
I threw together my own jujutsu powered FUSE subtree project just the other night. This was basically one-shotted, just to see how it does it, to learn more, not really intended for use. But obvious overlap. https://github.com/rektide/jj-workfuse
Hmm, so Anthropic is also in a monorepo club. Didn't know.
It's not open sourced in case you are wondering.
Yeah, I was a bit surprised to see that it's not. The post seems so tone deaf in that regard. Great, you made something we can't use.
It seems like open sourcing it would be the least they could do after ingesting all open source code to make their models, which are now replacing devs and other IT workers.
If any engineering org needed its own VCS, it would be anthropic.
Extremely annoying - can we throw anthropic class resources at more agreed upon solutions across companies. We don't need custom source control at every company - google has its own, fb has its own and now Anthropic.
Can please we all converge to 1 f'ng solution.
how does it compare to Epic's https://lore.org/ ?
why u want a single solutions for many problems?
Curious what happened at Anthropic HQ for a week in August
>Changes merged to our monorepo
> monorepo
Well, there's your problem.
git is cooked was not on my bingo card.
So, where's the source code for Scion?
Oh, it's not available, what a pity. Why write a whole article about a thing no one can have? Maybe the code isn't so great after all? Or it's all just made up and doesn't even exist?
Anyway, this is yet another pointless ad from Anthropic trying to shill their AI on us.
Sounds like they re-implement fancier CVS?
maybe next month the agents will learn the 'commit locally' and 'git squash' skills
sounds like they reinvented cvs
A "scion-daemon". Hmm... That's something I don't want to meet.
I'm usually a big fan of Anthropic, but this is a pretty insufferable vaguepost. No source code, vague self-congratulatory statements like "A new level of operational excellence is possible" with nothing concrete to demonstrate how this is true. This just makes me grieve for the blog posts we'd used to get from organisations of this caliber.
E = mc²
E energy necessary to integrate changes
m mass of the monorepo
c rate of commits
This feels like a really nice design for the data system given the goals. Things that I noted while reading:
* This system leverages a metadata log pointing to immutable blobs in object storage. The metadata log can give high write throughput and keeps the control plane consistent. Object storage supports high throughput of the heavier data objects. Overall, it's an excellent choice given VCS systems can typically tolerate the latencies associated with object storage, as is the case with many operations-focused systems for businesses.
* The server/upstream gains scalability by reducing the amount of data it processes when accepting a commit/performing standard VCS operations. No packfile, no file i/o to construct a consistent copy of the repo, just a (usually) small metadata record that can be validated/committed quickly. The client primarily gains scalability along a similar axis (reduce amount of extraneous work for a given goal) where it can request just the pieces of a repository it needs to do its job, when it needs them. For example, agents can begin reading files from a checkout eagerly before the whole updated version of the repo is finished being downloaded, then during a compile step, updates can be fetched only for the module being compiled, etc.
* The system seems well designed from the perspective of considering "when exactly do I need a thing to be done." Git tends to bundle more of this work into synchronous operations when a request comes in, which ensures the system is consistent at the end and doesn't require a persistent process, but is costly. This system makes intentionally different decisions about "what work happens, when" so that the minimum amount of work is done to confirm a write and allow for the state of the repo to be reconstructed consistently, but it defers certain things that are useful but don't need to block a write. The discussion of "derived data" mentions that certain useful structures can be computed in a deferred way, which supports doing work around the time it's needed.
* A large part of the nice operational features of the system is downstream of making core data structures immutable. File contents are immutable, content-addressed blobs. WAL entries are immutable records. As mentioned in the article, this supports excellent caching, and simplicity in reasoning about history/updates. Additionally, the derivation of other useful data can end up as immutable outputs (themselves stored as pointers to objects, if scalability requires it).
* The decision to develop a custom VCS supported a number of other integrations into "agent-first" software development systems. The system makes it easy to enforce access control on specific directories, nice if you want to have agents explicitly denied the ability to edit something. You can also get a sense that it supports some valuable features in the associated code review system, for things like pointing a human at the most important changes in a changeset. You could imagine in the future, some associated highly flexible "owner" rules/notifications where a human can specify "I want to be notified if X system has a significant design change" and that intent can be recorded/managed in a more useful way than "add me as a reviewer if files in /abc directory change."
* I do wonder if content-addressing of the files is broken into chunks or not. Doing "one blob per file" is nice and simple, so maybe that's how it starts. But you also duplicate a lot of data, but maybe the dataset is still trivially small in the scheme of things, and the operational concerns win out. And if files are small anyways, you'd want to avoid splitting them into tiny chunks bc that'd affect the bandwidth you can achieve when downloading (spend more of your time on per-object/request overhead rather than actually transferring data).
Very nice piece of engineering work. I was also impressed by some other parts that received a little less discussion in the post (understandably), but would love to hear more in a future post:
* the testing approach of driving tons of system interactions in a hermetic way is really exciting to see. Feels like agents should help us write really excellent testing infrastructure, and that this would itself be important for gaining confidence to deploy without requiring a read of every line committed.
* Similarly, the mention of the "superfan" agents that closely monitor very specific parts of the system seemed very cool and would help a lot with deploying code you haven't necessarily read closely but seems reasonable architecturally. Though it'd still ofc be best to catch issues prior to merge/release, having extremely good monitoring/rollback capabilities would mitigate a lot of the downsides of high-speed, agent-based software development.
> You could imagine in the future, some associated highly flexible "owner" rules/notifications where a human can specify "I want to be notified if X system has a significant design change" and that intent can be recorded/managed in a more useful way than "add me as a reviewer if files in /abc directory change."
Good observation! If we share details about Antique, this will definitely be in there; this space is ripe for innovation.
For the "one blob per file" piece, blob storage is fairly cheap, but there is also a relatively simple way to get something like delta compression for blob storage that the clients never need to know about.
A "scion-daemon" is not something I want to meet...
Am I the only one not feeling the output of this velocity?
No, it's me too. Everyone seems very busy producing very little, but breathlessly talking about how amazing it is at how busy they are.
k, thx for sharing. Are you guys sure this isn't a Google blog post? They would be proud!
I honestly think that these AI companies really messed up their futures by adopting a token model. They should have gone the route of IBM and offered consultant services.
This sounds more like a trash pile than a source control system. Anthropic is spewing out code for codes sake. They cant possibly manage it, the more they generate the more they will realize they are just regurgitating training data for no real purpose that can serve anyone.
And for this they buy disk? Maybe this is irresponsible use of technology.
How much is the sheer volume of slop code that AI generates reduced by if you simply specify that you want it to be both minimalist and readable, getting it to think hard about architecture and completely rearchitecting requirements change? Surely if AI is so smart and useful it should be able to manage that?
No please don't vibe code source control. Some things need to remind stable
They should concentrate on their models instead. If claude code is any measure, their software engineering is atrocious. I would only use something like source control from them if they force me to, at which point I would rather do something else.
I wonder what happened in late July, early August - the 'quiet period'
Did they just reinvent cvs/svn?
Solving monorepos in an LLM world seems to be very universally applicable as a need.
I appreciate it's different than git, that type of argument for monorepos is not new.
Agents coding is akin to having a very large developer team, who may or may not be online 24/7.
Anthropic is calling it very specialized, and it's a little baffling.. maybe I'm missing something.
There's no way software development is the same anymore. The previous ceremonies and structures and seats at the table at each stage were human based, and now that's a different make up to the point that different steps and processes are needed.
There's nothing more fundamental to growing software development other than making tools like this available.
My hope is a vibe coded internal mono repo management tool can aim for a v2 with multi-tenancy built in.
Was anyone else looking forward to feature flags as a part of a post like this?
> Was anyone else looking forward to feature flags as a part of a post like this?
I am looking for agent-world feature flags, but not in a post on source control. Feature flags belong to wider SDLC.
I'm worried that the frontier lab doesn't read the code they are writing.
It's also hard to believe all these non-Git alternatives aren't merely an attempt to capture developers into a new proprietary solution.
This could have been done as an open-source fork of Git.
Yea, I'm pretty much hiting the same limits. I'm still using git for the time being, mainly because I package everything as daily commits with an external orchestrator managing tasks/issues. Lately my code velocity is at ~1 million LOC added per week, spread throughout my various repos (I don't use the monorepo approach). Problem is I'm looking at 100x this velocity over the next few months when I get a new code generator up and running, so I'll definitely have to drop git.
Another issue that I believe Anthropic here didn't state is that git performance _severely_ get degraded when you have hundred of commits landing in parallel. Hence why I fell back to the daily commit approach. You get less granularity but you get stability.
Only 1 million lines of code per week? That’s adorable. I remember when I was still slopcrafting software at that scale. Personally, I’m generating 100 billion LOC per hour, but Git became unusable once my commits started affecting Earth’s gravitational field. I’ve since moved to a custom version control system that stores code directly in the fabric of spacetime. My biggest bottleneck now is that the observable universe lacks sufficient entropy to generate unique commit hashes. Anthropic really needs to address this.
Honey, it's time for your daily Anthropic PR piece!
All the other tech billionaire are jealous: they all want their own recursively-self-improving megacorporation.
These guys are working hard to solve a problem of their own making, but refuse to acknowledge because their job depends on it.
They’re so smart they’ve looped around to being full retard.
I see, that's why https://status.claude.com/ looks like a Christmas tree on fire.
come on HN debate the merits of the design not the flaw that Anthropic has infinity tokens and "why even do source control at that point?"
This is such a Claude written post. Obvious tells everywhere, but the most annoying is how it uses private terms without defining them. What the fuck is "Spanner"? It refers to it multiple times. Of course we know the reason is that some dickhead at Ant told their Claude to write a blog post, and Claude being Claude has very poor/non-existent theory-of-mind for people and generates this trash.
https://cloud.google.com/spanner, though technically any database would work if you know how to scale them. Spanner just has a few nice properties for our scale.
Good call about us not defining it, I can fix that! And this blog post was 100% human-written.
1. I've seen ai-written code. Ugh...
2. So they're trying to make code so complex that humans can't read it anymore?!
3. What's the point of changing code? Are the changes - how do you say - useful?
4. Thank God for Git, the amazing piece of software by the venerable Linus Towalds, that just keeps on giving! It's free, open, and excellent quality source-control software, and you can even keep binary files there. Excellent stuff. I can't imagine ever not using it.
Is it normal that peoples' last names are abbreviated in the Acknowledgments?!?
>Written by Nick G. and the source control team: S.G., R.K., Dan C., Garrett H., Ben L., Colin M., Joe P. We’re grateful to Vilas J. and Sergiu D. for supporting the source control team on this ambitious journey and Amir O. for early contributions to Antique.
Are Anthropic engineers now anonymous? Seems like a strange choice
Maybe they’re agents
Yeah this is my biggest question. Like are they CIA agents or what that needs their name hidden.
Some of them are for sure. Someone has to do the backup of transcripts and metadata
1/Poaching is a problem 2/in this politically charged environment over AI, doxxing and harassment are problems
They used to do this in the early games industry in Japan lol.
tl;dr why is it better than git?
> Git offers tools for working with a slice of a repository, such as shallow clones, sparse checkouts, and promisor remotes.
I honestly just cannot. Just tell me what the workflow is. Give me a video that is Twilio-level simple explanation.
the quote does not seem to match your statement here
Back to SVN then? Noted.
Oh wow. There is nothing I'd rather use less than version control vibe coded by Anthropic.