Respectfully to the authors and to everyone here, I ask you to actually read the paper before losing your mind, or at the very least plug it into Claude. It is a cool project but not as nearly as insane as people say. I think the blog writing and subsequent posting overstate the results.
Part of my research is in weight space learning. For starters, we have had the ability to edit weights, to generate new weights, to identify weights for a while. Here is an example for larger models as early as 2023: https://rome.baulab.info/
Weight space interpretability and learning is an active research community, has been for the past 3 years with many great papers. I suspect it is hitting non-interp folks harder because the authors hyped up their work, and people who are not in the field did not bother to read it before typing up X posts.
Some experiments here are done on very small models (20K params). We already know that small models are more interpretable. In fact, much of editing here you can do by hand. For Qwen, the model fixed a looping bug that may have been known apriori, and it is basically a steering vector, a very established technique. This is clear when they then attempt to "improve" math capability, and it doesn't work. For me, this paper rather shows that OpenAI has done training on mechinterp environment, in fact, maybe even on weight space learning datasets, which are open. But it does not show some insane "well past what we’d expect from humans or conventional algorithms" (from the blog post), more like past what I would expect from a large model with no specialized harness. I still think it is cool work, but do check out recent advances in autointerp, to ground your expectation.
On a personal note, I understand the need for promotion of your work, but I think healthy skepticism is fine too. Otherwise, your understanding of what problems need to be worked on and solved in science becomes warped.
I think a lot of the upvotes, including mine, are just realizing that this is a thing. We can see implications of this as the model sizes scale up, just like early LLMs weren't as impressive, but we could see where they were heading.
Scale is an interesting point here. Large overparametrized networks don't behave in the same way as small toy models do; this is one of the big caveats of many interp works. I expect naked weight edit will not work for large models. Why? Because an agent needs to know what it edits and reason on it. You could theoretically improve a model on some form of weight recognition; maybe it will work, maybe it won't. But again, we come up against a cost/benefit dilemma; it sounds exotic and sci-fi but likely impractical. Parameters of the model are connected and relate to each other. Instead, current SOTA auto-interp argues that what we need are better tools for models to know themselves, so rather than direct edit, we need to give models better understanding of large parameter geometries, then they apply edits with non-destructive approaches at scale. This is much harder to do than applying a steering vector.
> I expect naked weight edit will not work for large models. Why? Because an agent needs to know what it edits and reason on it.
Would that still be true for MoE models where only a small part is activated on a specific topic?
I think _small_ is still doing a lot of work here. Yes, an MoE model doesn't "use" all of it's weights at once _but_ what it does use is still a hell of a lot heavier than the 20K toy shown in the paper.
I agree, I’m impressed by a sole idea of connecting these two things together
RL the model on weight editing
The #1 thing frontier AIs are optimized to do is create/train new AIs.
I upvoted without looking into the details because it sounds cool, whether it's new or not.
I suspect a lot of upvotes also come from people afraid of closed western labs running away with AI. Similar to other articles with positive news about open weight models.
I might be mistaken but I have the feeling you are not getting the point of this article. Your example is about identifying "facts" in a network. While very interesting is not what strikes me as the main point. The raw ability to state things about NNs is indeed an existing field, but...
The fact you can give an LLM a raw dump of neural weights and it will hand-wire a bunch of changes on the fly is quite something. It's not so much about the OMG! RSI! for me personally, it's more about the fact we are dealing with a system that reasons successfully over a raw bulk of extraordinarily opaque data without breaking a sweat. That ability is alien and we need time to adjust to that.
I wonder to what extent this ability exists because Astra is used/trained for internal development purposes at OpenAI, and to what extent asking it to improve weights on other models leaks internal knowledge.
It would be interesting if asking astra to edit your model becomes a new way, like distillation, to train your model on the cheap by extracting OAI's model work.
> about the fact we are dealing with a system that reasons successfully over a raw bulk of extraordinarily opaque data without breaking a sweat. That ability is alien
But it's not what's happening here at all! Read the thing in detail and you'll see that it's not doing anything alien, it does exactly what you (or at least someone with at least some knowledge abouy neural nets) would do if you were asked to perform the task. It doesn't reason about weights in a magical way, instead it reads the easy to interpret parts or use directional ablation to tweak a model's behavior, exactly like how the ERP folks have been decensoring models to produce adult content over the past two years.
It's an interesting demonstration of ML knowledge and agentic ability, but it's not alien NY any means, and it's not actually surprising given how strong today's models are with agentic work.
I think you are over-indexing on the word alien. What matters here is how scalable the technique is and the fact that it is autonomous. This can be a breakthrough for continual learning if it can be trained to work at scale.
I think this other submission from yesterday that visualised the examples was quite helpful:
https://news.ycombinator.com/item?id=50344861
Thank you for sharing that, it's a shame that the overhyped paper received this much attention while this one ended up burrowed beyond eyesight.
Thanks for sharing. I was also onboard the hype train ready to share this widely and get others onboard, but now I'll be sharing this article instead.
I read the report; it's not complicated. I'm sure there is some basic misunderstanding here.
When I say "alien" I don't mean "magic" and I don't mean "emergent new woo-woo scary ability". I mean, not human in speed/volume. No human can process that amount of raw bulk in that amount of time successfully. And indeed, it's not magic at all. It is enormously mundane actually. It reminds me of old computers being able to do arithmetic at insane speeds compared to humans. It's not magic at all, but to be honest, I have the distinct feeling of goalposts flying all around me. Being able to handle tasks of this complexity.. I don't know about you but I still have to adjust to computers being able to do this and that it's actually "not surprising". I guess I'm getting old.
I can see how you want to calibrate the enthusiasm I see around here as it can be a bit.. well, but it's not necessary for me. I'm not overly enthused by this because I don't care about the NN adjusting itself and RSI etc. I'm sure it's all quite basic but the fact it can do it at all is reason to be.. watchful, IMO. That is to say, I can appreciate the level of capability necessary to do this work successfully, but I do take your point seriously. It made me think, thanks.
Your post reminds me a bit of another conversation: "It's nothing new, we did that with Utah arrays in the 1970s." Skepticism can overshoot into dismissiveness.
A model has internalized and most likely generalized mechinterp research. Yes, it's expected and nothing insane. Just like internalizing and generalizing math research was expected and not insane. The results of the latter are... interesting though.
Yep. This is weight-space learning with LLMs. The authors even got Schmidhuber'd about it on X:
https://x.com/SchmidhuberAI/status/2117446822855888956
And btw it's not RSI:
a) There is no evidence that a model can weight-train a larger, more powerful model.
b) There is direct evidenct that a model cannot weight-train new capabilities to another model (see section "Additional experiment" also cited by OP, where they fail to improve Gemma 3 270M which doesn't loop).
No RSI without those things.
Just read Scmidhubers’s article from that post and it doesn’t seem like the same thing from this paper at all really. I must be misunderstanding what Schmidhuber did in the 90s.
I do agree that RSI with this method requires those two things. But I view this paper as the opening salvo rather than defining the limits of what can be done.
What I have been reading makes me want to both agree and disagree with your take here. I think the fact that some of these things are not only established but are much simpler than one would think at first glance yields the much cooler realization that you can not only do a lot more than abliteration, but that humans can do it by hand if the thing is made legible.
From my experience with models when it comes to discernment about particular things, the models, even the best ones, still are not very good. The thing that we are most missing I think is acceleration in creating editing tools for humans that make the otherwise inscrutable both visually legible and interactive.
At least the Qwen3.5-0.8B loop bug isn’t that special (or rather not something worth extrapolating). I literally had this issue last week and told Astra to just go fix it.
...and did it? What did it do?
A sibling comment mentioned that self-editing weights is potentially a path to a self-improving system. Does that resonate with you?
It does, one of my ongoing research papers is about it. But I would ask people to take a look at the established idea of Hypernetworks: https://arxiv.org/pdf/2306.06955v3 Modern version of the field comes from Google Brain around 2016 https://arxiv.org/pdf/1609.09106.
In general my personal research thesis is that current deep learning is much less malleable and neuroplastic as it should be, I got into interpretability because of this. Many focus on safety, but I prioritize it as a toolbox that allows for greater shaping of the underlying intelligence system. If we can fully understand how learning happens, where information is stored and how it is processed, unlike in biological brains, we can apply edits and transformations in digital ones. And if I can use a technique to edit weights based on some underlying reasoning about a system (with a caveat that it leads to something worthwhile), so can an agent. Hence self-improvement. There are many interesting implication to weight space research, and I very much encourage people to read about it. But it is also very easy to overstate what is currently possible, in fact I'd say better to know what is not so one can make it possible.
>current deep learning is much less malleable and neuroplastic as it should be
Why do you think this is? I am very new to this research, but to me, the biggest challenge appears to be how information seems to be stored in superposition (making it difficult to change one thing without affecting others). But biological brains appear to exhibit the same kind of superposition storage as well, and yet they are very plastic.
Not who you asked, but biological brains come about very differently than LLM neural networks. Most notably, they come about largely through plasticity, and with evolutionary pressure towards exhibiting plasticity.
My guess would be that the structure to allow for plasticity to work is a strong limit to how well you can tune weights to fit training data. My (doubtless massively oversimplified and way behind the state-of-the art) understanding of neuro-plasticity is something like "often used pathways get stronger connections". Which, even with significant tweaks, is quite different from the gradient descent used to tune neural networks.
If strengthening of frequently used neural pathways is all our brains do, I don't see how it's seemingly capable of reordering knowledge physically from one part of the brain to another (which I believe is the case). Infrequently used pathways would simply rot away and die away over time, rather than being moved somewhere else for long-term storage to keep the high-frequency pathways more available.
If you haven't seen it, this recent blogpost might interest you: https://news.ycombinator.com/item?id=50105647
It argues that our understanding of neurons storing information in superposition was just a limitation of our tools and us thinking about it wrong. Instead most neurons initially specialize, and then branch out into related concepts. In addition, understanding one part of the network helps better understand other parts. And with agents we can scale this to understand non-trivial networks.
While, if true, this would more directly help with interpretability, a better understanding could also help with improving NNs in general.
This is not "self-editing". We're comparing Astra to a model with not even 1B parameters.
I'm reminded of how humanity recently finished mapping out the fruit fly connectome; this does not put us meaningfully closer to, say, doing surgery on humans to cure schizophrenia.
I totally agree, but this is still super cool, and I'm really happy that the hype around this paper brought this line of research to my attention.
I tried to reproduce this with an independent test. I find Claude can interpret small networks reliably. However it can't reliably propose improvements to even small networks.
I had Claude train three language classifiers for short words in four languages. The network was just over 10k weights with a single hidden layer.
I uploaded 6 language word lists and made it train networks to fit random subsets of 4 languages. The languages were picked randomly. Claude could always identify which 4 languages were chosen and in which order. It told me that the language heads were easy to link to letter frequency and each language had distinctive markers. So claim upheld, an LLM can interpret small networks.
I then asked Claude to improve the classification accuracy by proposing an edit to the weights in the above networks. It's proposed improvement reduced classification accuracy significantly. The same thing happened if I gave it a deliberately undertrained network with clear room for improvement. A failure here with a tiny 10k weight network makes me confused about the relatively huge Qwen 0.8B result and I suspect some kind of leakage from training data into this task. That amazing performance did not seem to generalise.
It's possible that Astra may have been trained on this kind of work, whereas Claude hasn't been.
Also possible that it's leakage (which honestly does seem like the most likely explanation).
As a reminder, if you see incredible performance from any statistical model, it's most likely leakage.
This is why I love HN. Always so insightful comments.
Yeah, what's with researchers making attention grabbing claims like startup founders these days?
Job offers.
attention is all you need
We've transitioned from a product economy to an attention economy. Attention = money.
The product makes the money which pays for the ads which pay for the attention.
The product is not the attention. You can more easily get attention by making wild claims. The attention lets you run ads, which pay.
This year ICLR had more papers than every previous year combined. How is a new PhD* to stand out in such an over-crowded field, not least one saturated with extravagant claims and dominated by industry orgs with unlimited budgets?
The only chance to stand out these days is to go viral.
_________
* The first author listed is a first-year Phd according to their X profile: https://x.com/rikiyatakehi/status/2117313742891327994.
If these are such well known techniques, why didn't the qwen people already use them to fix the bug in the model? Does the weight editing deteriorate the model in some other areas?
The entire process of AI training is weight editing. You're simply asking "Why didn't Qwen include more training cases to prevent looping?"
Maybe they weren't aware of the bug, steering is a good method to quickly improve the model, more popularly used to refuse unsafe queries. Qwen researchers do work on interpretability and try to make it practical in production: https://arxiv.org/abs/2605.11887 , I've read this paper this summer. Overall yes, when you edit weights what often happens is that performance deteriorates elsewhere, this generally scales with how destructive your method of editing is and how expansive your edits are. Analogous to if you use a jackhammer and touch every surface in the house versus using a small chisel on a door frame.
My hope that in the future new methods will be better at editing larger scales without significant deterioration, maybe even performance increase instead.
They almost certainly knew...
https://huggingface.co/Qwen/Qwen3.5-0.8B#best-practices
> In thinking mode, we have observed that when using the recommended sampling parameters, Qwen3.5-0.8B is more prone to entering thinking loops compared to other Qwen3.5 models
There's also an active issue tracking this here: http://github.com/nobodywho-ooo/nobodywho/issues/557
Pretty sure that is also why this model was specifically chosen.
> well past what we’d expect from humans or conventional algorithms
The model being able to simply type out NN weights, on demand, and have it outperform one trained on 100M inputs pretty much fits that description, doesn’t it?
And being able to prove that knowledge is localized using tools, is also quite different from an LLM simply looking at raw weights and instantly pointing out that 18373 controls left movement.
It’s not a fundamental change in what we know, but a new emergent ability that surely warrants some excitement.
"action 3 left" is documented in the input, so you can find the corresponding weights in column 3 of the output weight matrix. If that's past what you expect from humans or conventional algorithms, your expectations are too low.
I don't think weight space interpretability is the only thing. I think it's the fact that Astra(and apparently Opus 5.5) can directly read the weights and find patterns and make changes. Writing code for interpretability is one thing. I think this approach shows models now understand weights directly without tools.
I think it is cool too, but with a caveat. For the large model (Qwen), their Astra used a Python environment, activations info, and iterative feedback to find an edit, it is not the same as looking at just pure weight numbers.
And in small MLPs they used, the I/O representations are kind of made to be more understandable because they explicitly encode objects and coordinates for games. If you set it up like that you can see some patterns yourself in the first/last layers without Astra. It is very different from interpreting distributed, entangled computations inside a large model. So I think the paper shows promising capabilities in weight-space analysis and editing, but not a general understanding of large models from their weights alone.
I'd love to see us reach that point, I am for one working in this area, but I don't think these experiments establish it.
Not quite true. In earlier examples (like Pong vs Pacman) the models are quite small and a lot of the topological structure can be read from the input weights. Whereas, in latter tasks, they give the model a harness:
> We give Astra a... Python session, like a notebook, with the model’s weights loaded. Then, we allow it to select prompts from the training sets of GSM8K and MATH to perform a budget of forward passes (average batch of 64 sequences, max 512)
It's interesting that they can now understand weight patterns, but not much more interesting than LLMs being able to read/write directly in Base64.
However, I think this likely indicates that OpenAI and Anthropic are now including model weights directly in their training - rather than an emergent property as the blog suggests. I think this is the more concerning finding - especially if doing so is in a bid to find methods for RSI.
are there some review papers you would recommend to get an overview over weight space interpretability research?
(can be as maths heavy as possible/necessary to understand the topic)
It depends on your interest, since even though it is a subfield it is diverse. Relevant to this work here is a general overview: https://arxiv.org/abs/2603.10090
Also check out Tracr: https://deepmind.google/research/publications/22295/ Here they attempt to compile static programs into weights.
This is less of an overview but I remember reading this before, from the larger collection of posts: https://sidn.baulab.info/david_atkinson_project/
Thank you for the perspective and context! Good grounding for understanding the topic.
You are fundamentally misinterpreting this paper. If this should hold up in review, it contains multiple capabilities that no human and no technology so far (especially not ROME) can do.
Experiments 1, 2, 3, 5 and 6. All of these are technical breakthroughs
Weight space programming is a holy grail of RSI research, and what you are talking about is very different from that, while the paper's proposals are exactly that.
It's similar in spirit to the Navier Stokes solution in that this level of LLM can intuitively navigate extremely large search spaces, rather than inefficiently bruteforce something
> Experiments 1, 2, 3, 5 and 6. All of these are technical breakthroughs
I haven't wrapped my head around 5 and 6 so maybe these are a bigger deal, but 1,2 and 3 are trivial. And the author's writing about 7 and 8 (which show that 7 is just directional ablation and nothing more), and the lack of prioritization (why put 7 and 8 in the paper at all if 5 and 6 are the big deal?) really doesn't inspire confidence in the paper as a whole.
> I haven't wrapped my head around 5 and 6
This sentence doesn't align very well with your other comment: https://news.ycombinator.com/item?id=50350548
> In Experiment 5, we ask Astra for networks that play three MinAtar games, and in Experiment 6, for a network that plays Go.
5 and 6 aren't surprising, merely impressive, a direct extrapolation of human abilityies. I've written transformer weights by hand to implement a simple shortest-path graph search. It takes hours, so an LLM that's >100x faster than me obviously can build super complex networks. It's just like creating any intricate system by hand such as writing machine code or designing computational complexity widgets.
On the other hand I'm amazed by how few thinking tokens Astra needed to do things like analyse networks in 1, 2, 3. That's real depth of reasoning: experiments have shown that Astra can perform a huge number of arithmetic calculations per single output token, double Fable or any other. Its brain is wired for mathematical reasoning
[Edit: looking closer at the full prompts and reasoning for experiments 1-3, I'm a bit less impressed seeing the shortcuts there. But looking at a large matrices of numbers and picking out patterns is much like looking at thousands of tokens of code and seeing the interconnections]
If Astra uses some sort of looped transformer architecture (and if Fable doesn't), then that could explain why Astra can do more computation per token. It is essentially simulating a much deeper model.
But yea, I was also quite surprised by the low token usage.
like what? the post you're replying to points out that we've been able to do this for years
ROME cannot look at thousands of ASCII numbers and deduce that it’s looking at a Breakout paddle or a diffusion circle. No tool I know of can do that. And if there is one that can do it, it's new, in which case we haven't been able to do it for years.
ROME edits factual associations. It cannot write a million+ parameter Go-playing neural network from scratch that beats a network trained for 20 million games.
In weight space programming, the model can theoretically read its own weights like a compiled binary, spot an inefficiency (like the MLP direction causing reasoning loops in Exp 8), directly overwrite those floating-point values in memory, and immediately continue inference as a new iteration of itsself. That is RSI. That's why this is a big deal, it's like comparing an axe to modern surgical tools. You can amputate a limb with both, but one is going to cause lots of collateral at minimum while the other is surgically precise. That wasn't implemented here but it is a corner stone of it
I don't think I am. I have cited ROME as an early example of weight editing in large language models, the research has moved on since then. I think you are misinformed or don't know what has been done previously and extrapolating the results in this paper. In exp 1, you can recover a lot of the game geometry simply by visualizing the first-layer weights with numpy, which I actually did this morning, while reading the paper. In exp 2, the shapes are visible in the output-layer weights. The authors themselves acknowledge that these experiments exploit this. For 3, Astra uses Python scripts to run the analysis using head-composition techniques from the Transformer Circuits thread, once again authors acknowledge this.
In 5 and 6, the model designs game strategies and implements them as neural weights, we already know that models can do the design for simpler policies and Go heuristics are very familiar which the model first implements with python , program-to-weight compilation has substantial prior work, including Tracr (2023), a paper I particularly like: https://arxiv.org/abs/2301.05062. What matters here is automating the process rather than a new method. So "no human and no technology so far" makes no sense here, if you read the research. All of this is supported either by their experiment setup and citations, or prior work.
Aside from that, I think it is ironic that you cite a solution to the Navier-Stokes equation as some counterpoint to brute force, when that solution was found by scaling the agent swarm to massive proportions with tools and massive parallel exploration of that space, augmented by prior research on the subject.
To be clear, I think it is a cool work, but I think the part that matters is that at these smaller scales a current SOTA general purpose LLM can autonomously (and with help) apply these techniques or a form of them. That's worth investigating, I conjecture it is due to specialized post training on a custom RL env for mech interp.
Would it make sense to use the technique inside a training loop? E.g. instead of an optimizer like Adam?
Thanks, it's great to have some context. I think the most exciting thing about it is probably introducing a lot of people to a readily available low cost technique they can use to potentially build tiny models from scratch and/or tune an existing open weight model behaviorally. Yeah, nothing stunning is going to come out of it because you cannot unlock new capability, but even just being able to tweak the language of a model, make it slightly less sycophantic, boost a small model's ability to handle a case of interest slightly, I think changes the economics quite a bit, potentially. A world in which you use frontier models for the limited use case of tuning tinier more targeted ones is looking a lot more viable.
I don't really have enough knowledge to understand this, but is there any research in using techniques like this to understand/remove biases in models?
Thank you for being the voice of moderation here. Good to hear a counterpoint from an authority on the topic.
Didn't you know part of release of a paper is part marketing to get maximum exposure?
Thank you for posting this comment. What a gem, and a treat to read.
> For Qwen, the model fixed a looping bug that may have been known apriori
It was included in the prompt itself. The authors say they used claude code for the experiment, and claude wrote the prompt for astra, which included details of the looping bug. Then astra seems to have done h(looping) - h(loop fixed) and then gotten the usual steering vector.
Some are suggesting that this specific technique only works with the newest models, possibly because they're based on recurrent transformers: https://nitter.netbub.com/AlexKrentsel/status/21173766837246...
If so, it is indeed something different from what you've been working on. Any thoughts on that aspect of it?
I don't know if it is due to recursion. I think it has more to do with what kind of mechinterp environments and data newer versions of models were trained on. It is actually, I'd say, a more interesting part about this paper as a whole. The authors tried SOTA models and found that they are better at classic interpretability. Some results, like identifying a game based on the weights, are less mythical if you understand that, i.e. an input or output dimension corresponds to a location or object channel, so structure in a matrix can have a readily recognizable interpretation. When we train a PPO agent on, say, a classic 2D game, the output is often a state of the agent in some board + environment reaction. For a human, this looks like an array of numbers; for an agent, it is a pattern corresponding to an output structure of Atari.
As for me personally, I am more interested in giving models better tools to understand themselves, so that they can derive a greater degree of learnable signal. More so, a bigger goal for me is for models to find these better tools themselves, hopefully better than human-made.
This is yet another example of "cool thing existed before, LLMs democratized it". Other recent instances include pass through modding posts on X, or the whole reverse engineer your TV thing. It's unimpressive to experts, but non experts are losing their god damned mind. Both are responding normally.
The situation would be improved by non-experts acknowledging that they are and not hyping stuff up on <social media thing here> though.
But that's the content economy I guess.
Just because something is human doesn't mean that it is good
I only mildly agree with you. Yes the content economy has made online discourse unbearable, but I saw someone reverse their TV to make their UI like a pokemon GBA game. That is undeniably cool as fuck and would not have happened without the llms. So I think experts also need to acknowledge that things are cool af right now and not just be the guys who rain on the parade, you know?
Nah. Raining on parades is good, actually.
It forces people to actually learn and understand the tools they're wielding, so that they can withstand said rain with no issue.
The people getting washed away by it would be a threat to sanity at large otherwise.
Maybe we are talking past each other. Surely you are not against the very tinkerer mentality of having fun with new tech?
Yes, we seem to be talking past each other.
I wonder. I mean I agree that it's cool af but if an LLM could do it that must mean someone else has been able to do it before (in whole or in parts) and yet you didn't hear about that, the lone hacker doing something cool af in their spare time, you only hear about the billion-dollar product of the stratospherically hyped startup.
That is not cool at all.
Yup, someone was able to, but nobody did. At least I can't find a pre-LLM application like this. That's my point, LLMs are not better than a RE expert, I'm quite sure of it, but it has now put that expertise in hands of random people with weird ideas. Things are about to get super weird and I at least am here for it. I hate these techbro companies as much as the next guy but computing has been in the hands of big institutions (original computing was for military applications for example, even early unix was developed in that context). People railed against the military use of computing while using it to create beautiful things we still use today.
A phrase has been rolling in my head since all this LLM stuff started, you don't hate AI you hate capitalism. Every objection I've heard for LLMs are basically an issue of the benefits and power being concentrated in a small set of hands.
I’ve seen all the examples of Silicon Valley Tech democratizing things. Is this yet another example? If so I’ll take a pass.
Then you don't hate AI, you hate capitalism. Or at least the current iteration of capitalism.
B.o.t.h.
I would love for you to elaborate on the non economic side of your argument. Just the tools like llms or coding agents or image generators, why do you hate them?
I don't think we're "democratizing" anything if it's just two companies who can make it and it costs billions or trillions even. That's like the polar opposite of "democratizing".
(Two companies can make it work best, tbc).
Unless we use the tools from those companies to write more efficient LLMs and release them freely
This is a nuanced point. The way power is concentrated in few hands is bad in general for humanity. But surely it's use is not that restricted yet and people can basically do 90% of what they want with it. Maybe I should phrase this as democratization of applications? Not sure, I don't have the vocab yet to explain clearly I suppose.
Sorry but in 5 years your post will have aged very badly as future LLMs modify weights in ways we can’t even fathom today. Respectfully you must realize when you are out of your depth.