> The White House and intelligence officials had pushed forward a classified contract between Anthropic and the N.S.A., which would allow the spy agency to use the company’s technology for a variety of purposes, including intelligence analysis and detecting new computer vulnerabilities.
Ironic that both sides are playing a horse shoe game:
Gov: The model is both a supply chain risk and also we'll DPA you if you don't give it to us.
Anthropic: The model is both like a nuclear weapon in terms of national security implications and safe for general release.
I mean graphite control rods do exist in nuclear reactors to absorb excess neutrons, preventing the fuel from going critical & making it technically safe for general use (THOUGH of course disasters have happened)
Or, alternatively, it may suggest that the NSA’s classified systems are not very secure, which seems at least as possible: they may rely on requiring physical access to these systems to even attempt to penetrate them.
Yes. The original curl post didn't say anything like "mythos sucks" but rather "it's only a minor improvement in comparison to already widely used models".
Which I think points at Mythos not being some big jump in capability finding things earlier LLMs didn't, it seems to mostly come down to massively increased compute budget and they finally catching up in context sizes.
Curl is such a small utility, and the effect of any single problem is limited.
Mythos's great strength was finding multiple vulnerabilities and chaining them together to break a whole system.
Look at it like this: It found one confirmed, minor vulnerability in Curl (but I don't think they have said what it was?). In another system that used Curl it's possible it could have exploited that vulnerability to chain to another, bigger vulnerability that was normally inaccessible.
'Mythos “broke into almost all of our classified systems, not in weeks, but in hours.”'
And the government's response was to limit access to US citizens? I don't believe this for a minute. If Mythos could actually break into all these systems, the government would declare it a national security risk and it would never see the light of day for anyone outside government staff with security clearance.
additional context from the article regarding that particular statement:
"[the statement] was oversimplified... In reality, the tests involved “red teams” of N.S.A. analysts who were using Mythos in a highly tailored environment that would be extremely unlikely for an adversary to replicate, officials said. The red teams began their tests within classified N.S.A. systems designed to be accessible only from certain computers and completely cut off from the broader internet.
The tests found that Mythos was able to identify cybersecurity flaws within that classified network quickly, but it did not actually break into those systems, the officials said."
mythos allowed mediocre people to get results by holding their hand through the process, or just ignoring their irrelevant input and knowing what to do.
if you throw millions of tokens at IDA Pro MCP with the right prompt lets just say security by obscurity fails miserably because there is no obscurity when the LLM chews through the decompilation.
>mythos allowed mediocre people to get results by holding their hand through the process
Isn't this what technology progress looks like? Industrial tools allowed mediocre people to improve their productivity by orders of magnitude which is how we managed(in the past) to build so many amazing things with less human toil and suffering than previous generations.
It depends. Mediocre doctor in a remote area with right tooling assistance as opposed to no one being available for someone who urgently needs one? Yeah this should be a thing.
Should a software bro in NY perform it in dark alley despite having best doctors few blocks away? Maybe not…
I'm sure many 'mediocre' people perform heart surgery. Only 100 years ago, the idea of a person without a certain surname or race, would've been a ghastly preposition, no?
Do you... think heart surgery has become LESS dependent on surgical skill in the last 100 years? Cardiovascular surgeons spend MORE time in training now than they did 100 years ago.
Did heart surgery as we know it exist 100 years ago, or are you trying to conflate things to make a point?
"heart surgery" isn't a technique". Name something, literally anything connected to the profession, and tell me whether the training time is naturally bound to keep going up and up.
Privilege enables you to rent competence, historically by paying other people. The slop companies will now sell you a simulacrum of competence by the token.
The fact that competence can (could?) only be acquired through sustained effort over a long period of time is (was?) levelling the field.
Selling simulated competence perpetuates privilege, instead of dismantling it like you seem to claim.
Perhaps, but I think volcanic eruption followed by system collapse is very compelling. Here is the story I find most convincing from the experts whose works I have read.
It likely started with a volcanic eruption, leading to widespread famine. Those in western Europe who didn't want to starve migrated en masse, as whole families, becoming the sea peoples. The powerful empires struggled to feed their people, and many were destroyed by the forced migration from the sea peoples. Egypt barely survived, but only as a shadow of itself. Many of the others were destroyed by those who had survived on marginal lands and didn't need complex societies to keep themselves fed.
Iron can't be the cause, as iron weapons pre-existed the Bronze Age collapse. I think the evidence is stronger that the collapse forced widespread adoption. The collapse devastated long-distance trade networks, which cut off the supplies of tin needed to make bronze. The scarcity pushed people to rapidly improve iron smelting.
I'm not a professional historian, but I do find the topic interesting. We should try to learn from past disasters to prevent repetition.
See Eric H. Cline's
"1177 B.C.: The Year Civilization Collapsed";
AI marketing bullshit stunts are unlike anything I've seen in 30 years. It started with MS Copilot so called capabilities for work, which were completely made up use cases that didn't work at all (3 years later still). We've had OpenAI "AGI is coming" and "AI will take your job", now we have Mythos being so "dangerous" for cybersecurity, which of course makes the average Joe interpret it as Anthropic being "the better overall company, the NSA uses it!!". I mean gov foes with Anthropic are probably true, but the marketing is to blame not Mythos capabilities. This is all so fucking pathetic
Well the good thing is you’ve done the homework to definitively demonstrate that this is remotely true. These confident claims of this all being some sort of unprofitable Ponzi scheme, not understanding the concept of a growth phase which a multitude of highly successful tech companies have already demonstrated work while simultaneously commenting on a site with YCombinator in the url are just getting amusing now.
Of course this is a profitable technology, and it doesn’t matter if any of the labs are profitable today or not. Running at a loss is a perfectly rational strategy.
(It's actually probably more profitable than their projections here calculated because they were expecting to be running Fable but can't, and Opus costs less to run)
We should seriously reframe this whole AI thing to "SI = simulated intelligence".
It's google in a box. Great achievement, makes knowledge work faster, but please stop bothering everyone else.
The Uber and Groupon people became billionaires, so the "Simulated Intelligence" folks will also achieve it. No need to worry and drown everyone in these bs stories only non-tech people believe.
Heh. In the Schlock Mercenary universe, "SI" means "synthetic intelligence", which is a level below real AI (which means what we would call AGI). And, as it says (in https://www.schlockmercenary.com/2003-07-21), SI translates to "kinda stupid".
Can you describe your experience using modern AI tools that led to this conclusion? It is hard for me to wrap my head around how my perception could be so different from someone else in presumably the same or similar profession. I'm not asking this in bad faith either but I think your getting downvoted because your comment comes off as a pretty strong assertion without giving details on how you got there.
A lot of effort is spent to make the "conversation" feel just like a human-to-human interaction. This is not a naturally occurring phenomenon due to the technology, but rather a feature carefully engineered by those companies in order to get people hooked. Then they have all these tiny nudges like the typing animations or the "thinking..." popups before the next chat message appears.
At some point you might have also noticed the over-use of emojis, the bolted-on jokes, and the tendency to always approve what the user says (even though they have toned that down after backslash). At some point too many people thought they were in a relationship with the chatbot, because it always encouraged and approved them, so they had to hotfix it.
It's a bunch of really dark psychological patterns that are carefully combined by very clever people in order to create the false illusion that the user is experiencing something deeper than an engineered simulation of human interaction.
I think the technology is really useful, but they are obviously not happy with simply replacing a google-like query interface, they want users to fall in love with the product and mentally treat it like a fellow human being - and that's what I think is insincere.
Not a good comparison. Education is the part where they train on all digital content they can get their hands at, no matter the copyrights.
You get your education, you can replace google as a query-response interface to all digital content.
But then they use system prompts to simulate a fake persona and a user interface such as female voices or chat conversation in order to suggest that one is interacting with a real human being. This is clearly aimed at exploiting vulnerable cohorts of people, because the knowledge base part of this innovative technology is already solved.
Like casinos and social media companies, they know the profit is in the "whales" who can be psychologically manipulated to spend their time and money against their own interests.
How would you program a LLM so it gives useful information to people with the least amount of people bitching about it?
At the end of the day the LLM does not have a native persona. It has countless numbers of them. It can act like an autistic man, a flirty woman, a kid from some country you've never heard of. Bringing forth an agreeable persona from the myriad is a bad thing?
To get more concrete are you using coding agents like Claude Code/ Codex/ opencode etc? What kind of work are you doing specifically?
If you are doing the kind of median enterprise tech work these tools are just good enough to do it at a relatively high level or atleast heavily augment people doing it.
Examples would be like adding routine CRUD features to APIs/ improving observability/ adding tests or accessibility features to codebases etc.
It try to explain it better in my longer sibling comment. I'm not using any coding agents. Their engineers can't be bothered to design their own webapp properly so I don't trust their binaries.
For me both Claude and ChatGPT are query-response services and replacements for google. They help with error messages, single-file MVPs, and software design problems such as comparison of different modules.
In my experience everything that goes beyond 200 lines creates issues down the line, so I try to keep interactions really short. Of course they can convincingly add CRUD functionality or tests, but one needs to double check their correctness, and if the subtle bugs are finally spotted then one needs to fix them anyways.
It's good for a first draft but I wouldn't use agents on a codebase I actually care about.
Unfortunately the billion-dollar funding forces the AI startups to make a return, and they are finding it in a vulnerable cohort of people who respond positively to a simulated human interaction, which is why they are focusing so much on it.
The query-response knowledge interface was the moat of google, and nowadays it can be 80% replaced with a local GPU and an open model. They know it, which is why they try to hook people on the simulated human interaction aspect of their interfaces through chatbots and voice chat.
POV OpenAI, early 2021. You have a pretty good next-token predictor called GPT-3. You noticed sometimes it can do useful things if you write out the start of a task and let it predict the next step. However, sometimes it's very difficult to frame a task that way. So instead you train it to predict the answer based on a question or instructions. Oh wait, it didn't get it right the first time... better let users iterate too. Now you have a conversation.
Things like loading indicators are basic good UI dating back to the 90s.
A/B testing and generally following user preferences might still push towards the dynamic you're describing, as it did with gpt-4o. xAI and a few other companies like Replika also intentionally created "companion"/porn AIs. But in general, natural language was previously exclusive to humans. It's completely natural that the first technology capable of it would therefore be perceived as more human. It's worth trying to resist this tendency, but it doesn't require evil intent on the part of the creators.
The "conversation" interface is exactly the same workflow as one used with google search: user states a query, page loads, user adapts query because they are not happy with the result, until they end up with a suitable result which makes them close the tab.
So they have made this amazing query-response system which is far superior to google due to the summarization of query results from the global web and the auto-translation to present them in the user's native language. This is the type of raw query-response capability which many software engineers are trying to use in their agentic coding sessions.
However, after achieving such innovation, the AI startups consciously choose to apply social media KPIs to their query-response startup, which incentivizes all the dark patterns we have seen in their user interface. They notice that a certain subset of users can be tricked into believing that the startup's query-response interface has human-like qualities such as a name and persona.
This user cohort shows amazing metrics in terms of time spent on app, so they adapt their user interface and their system prompts accordingly. The AI startup doesn't have to care if the reason for humans accepting the illusion of a simulated human interaction is due to social circumstances (lack of emotional intimacy) or an underlying psychological vulnerability that the startup is actively exploiting.
The AI startup only cares if their "simulated human interaction" product receives negative attention from normal people who are not part of the vulnerable cohort, e.g. the suicides or the parasocial romantic relationships with the chatbots.
It is exactly the same as in the gambling industry: There is a certain subset of users called "whales" who are the cash cows for casinos, but if you look at the actual humans who are labeled with this term one can see pathological gamblers, most of which are ruining their lives and families. Casinos do everything to prevent people from jumping from their roofs after they lost all their money.
If AI startups can use simulated human interactions to make vulnerable people act against their own interests in the same way as casinos and social media companies do, it will allow them to make shitloads of money.
But if you're actually a clever person then be honest to yourself and others about what you are working on, and why these human-like features are really added to the user interfaces of OpenAI or Anthropic or the other AI startups.
So this is my framing of the situation.
I don't think this kind of problem can be overlooked by the insiders, and we might see some internal rifts along these lines: Will our AI startup simulate a human interaction in order to exploit our vulnerable peers, or will our AI startup focus on delivering the best response to our user's queries?
Because now we have local models, which - assuming one has suitable hardware - provide 80% of the utility in terms of a query-response knowledge base.
As we are currently seeing, the AI startups with billion-dollar funding have very big economic incentives to focus on the "simulated human interaction" part of the equation, because their investors need returns.
The biggest strategic blunder I see at Google. Because if Google actually changes their excellent query-response user interface to a chat conversation which simulates a human interaction with persona, name, and voice, then they knowingly pivot to the same social media KPI driven business as OpenAI and Anthropic are struggling with.
Feels like OpenAI and Anthropic are both more interested in B2B. OpenAI I'm more suspicious of after GPT-4o. I mostly use Claude and I haven't noticed anything that feels like an intentional dark pattern. It's constantly reminding me that it's a chatbot.
I was able to identify, diagnose, fix, and upstream a minor bug in and erlang/OTP ssh key implementation with Opus in maybe 20 minutes (+2 weeks or so for upstream). It is not impossible that I could have done this before, but it would have taken days or weeks. The actual fix was about 2 lines of code, hardly AI slop, but getting there would have been quite the slog, and I never would have done it.
There is a lot of the reason for AI skepticism out there, but people tend to do massive overcorrections and underestimate the force multiplier it can be, particularly for people with some idea of what they're doing and a good grasp of how to take advantage of the tool.
So let’s say you’re in Anthropic’s shoes. You see that LLM’s are getting better and better, and it’s very possible that they will have some impact on jobs in the next few years, and a very meaningful impact on cybersecurity.
Is it more ethical to stay silent about these concerns, as you might have a bit of self interest? Or even if it looks a bit self interested, is it better to warn people ahead of time? I think the latter is obviously the better position.
The issue is both OpenAI and Anthropic have lied so many times that it’s no longer rational to take anything they say at face value.
Also: they don’t have to know they’re lying to say things that aren’t true. There is definitely some cult-like behaviour at the moment on the west coast
Be specific, what do you consider their lies to be? Also, this is pretty straightforward. You have a decade of extremely stable and predictable performance trajectory. It’s easy to see the writing on the wall. You can feel whatever which way about their motivations and ethics but if you read say Dario’s raw words they are pretty reasonable. We have to have a good regulatory framework and do what we can to prepare ourselves while also not ceding a critical strategic advantage. The west coast is always cult like, that’s not new. And it ignores the very real substance to the discussion.
Every year since 2023 the models are too dangerous to release and in 12 months all white collar jobs will be obsolete. This might not have been a deliberate lie but it's clearly been untrue and they've said it again and again.
Predictions with wrong timing are frankly worthless. I predict at some point in the future the S&P 500 will be at 10,000. Of course I'm guaranteed to be right. But have I really predicted anything useful?
If Dario was really worried about protecting the sheep, he wouldn't cry wolf every five minutes because everyone knows that's the worst possible thing to do.
And if you want to ask if Altman is trustworthy... ask Satya Nadella or anyone else who's ever made the mistake of doing business with him
> Every year since 2023 the models are too dangerous to release and in 12 months all white collar jobs will be obsolete. This might not have been a deliberate lie but it's clearly been untrue and they've said it again and again.
How is a prediction a lie? Did they tell you "this will definitely happen in X time"? Their speculation is not only valuable (they are the closest to the technology) but also necessary (they need to buy long term compute contracts so these predictions are literally what they have to bet their real money and company success on).
They have said again and again that this will make an incredible amount of tasks obsolete, and they are of course right about this. The models _are_ dangerous to release, every time we hit the frontier. This has become _increasingly true_.
> Predictions with wrong timing are frankly worthless.
Who cares?
> I predict at some point in the future the S&P 500 will be at 10,000. Of course I'm guaranteed to be right. But have I really predicted anything useful?
You aren't cherry picking and strawmanning here? Should we have a tour of all of the things that have indeed been predicted well and already come to fruition? Was 2025 "the year of agents"? It very much was, wasn't it? Additionally, unlike the S&P, performance trajectory, for almost a decade, is incredibly stable and predictable. It's hard to know, a priori for a given task or category of tasks, what specific error rate will trigger a phase transition but it's absolutely obvious and clear that this will happen quickly. It has indeed happened quickly. Does 2026 coding look anything remotely like 2024?
> If Dario was really worried about protecting the sheep, he wouldn't cry wolf every five minutes because everyone knows that's the worst possible thing to do.
No you're right he would make well reasoned arguments for the types of problems we need to address urgently. Hmm...that feels pretty ethical.
> If Dario was really worried about protecting the sheep, he wouldn't cry wolf every five minutes because everyone knows that's the worst possible thing to do.
I don't feel either of them are trustworthy, they are CEOs acting in their companies best interest. But people suggesting Mythos delay was some sort of PR ploy is some of the most extreme mental gymnastics I've seen. I listen to the actual words spoken by these people and consider the hard data that is in abundance at this point. I listen to the large body of research on alignment and safety and measurement that anyone can read for themselves or use AI agents to digest for them.
I’ve just watched enough old Adam Curtis documentaries to know historically how these things always end, true believers in many dead ends have exactly this kind of zeal.
Very smart people, reasoned arguments, “science”, all wrong.
I think that Anthropic is fully absolutely unethical. And they lied a lot. They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.
If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
It really does not matter how much they believed own doom predictions, because they were actively trying to make them true whether realistic or not.
> They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.
These words make no sense. Anthropic delayed mythos/fable rollout. A mythos model without safeguards would have been a pretty bad idea, and they sacrificed a ton of revenue and risked being scooped by any of the other labs in the meantime. Frontier models are only frontier temporarily until the next lab releases their model. Of course they are a company and need to act in their own best interest.
It is also clearly serious the problems we need to think about as we march quickly towards even more capable systems. Why on earth is it a problem to point this out?
> If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
What a really weird take; they employ some of the best safety and alignment teams in the industry and this is an active area of research that they are campaigning for more attention on. You complain about them “doom trolling” and then complain they don’t do anything about…the doom? No sense at all.
It is perfectly consistent to (1) sound an alarm and (2) March full steam ahead as quickly as they can. If they don’t do (1) that’s unethical. If they don’t do (2) someone else will. I would rather someone like Dario align these models than the CCC. Plus it would be nice not to have a war over Taiwan which is inevitable if China gains enough of the upper hand in this AI race.
Be specific, what are you talking about. Industry has been continuously warning about many of the complex problems that are going to happen as a clear consequence of the technology. I don’t know of any problem they have talked about that hasn’t either already come to fruition in one sense or another or that just hasn’t yet arrived. Dario has been predicting the end of coding for a long time now and look where we already are.
So yea no it’s more like it’s important for industry leaders and those closest to model development to proactively identify the issues that they don’t have complete control over or that we don’t have a regulatory framework for.
Super puzzling to see these comments and of course with zero specifics just “they’re all liars and grifters”
I'm talking about the breathless alarmism that Dario and his company push out as a marketing strategy. They've given us such gems as these:
- "It’s a bit like selling nuclear weapons to North Korea" (from the company that can't go more than a day or two without serious downtime)
- "We are releasing a model that is too powerful for the public"
- "It would be good for the world to have the option to slow or temporarily pause frontier AI development."
- "I believe that biological risks may soon follow, and that serious AI autonomy risks may not be far behind."
You can fill my ear with nitpicks about there still being time for these cries of wolf to be born out, but be prepared for me to wax philosophical about all things being possible given an eternal timescale.
> Dario has been predicting the end of coding for a long time now and look where we already are.
Where? It seems exceedingly unlikely that developers have all been phased out while I wasn't looking, as Dario prognosticated. And even if they all up and disappeared, AI still hasn't found a toehold outside of the relatively niche market of agentic coding.
Are we really saying that Anthropic claiming AI would take over industries was some benevolent ethical move rather than marketing their product as a cheap replacement for human labor that works in any industry? Wouldn't the ethical thing, if they were actually concerned about labor displacement, be to shut down the lab and work to disrupt and disable other labs instead?
Oppenheimer believed that technological progress is inevitable: if something can be built it will be.
Anthropic (and Deepmind, and some at OpenAI) believe the same thing.
Their ethical argument is:
1) This technology is coming whether or not our company does it or not.
2) Strong AI needs to be under human control, and we are the best placed to develop techniques to make this happen.
To be very clear: Anthropic (at least) is very happy to restrict access to their best models. They have continually campaigned for regulation to make sure others have to do the same.
> Wouldn't the ethical thing, if they were actually concerned about labor displacement, be to shut down the lab and work to disrupt and disable other labs instead
Personally I strongly reject the idea that labor displacement is unethical.
It will be a serious problem to deal with, but that doesn't make it unethical.
The steam engine displaced labor. That doesn't make it unethical.
What are those to be created jobs going to be doing that AI won't be able to?
There's two big differences with the steam machine: this change is happening much faster so society has much less time to adapt, and it's got a much wider scope. Steam machines only replaced a small category of jobs.
The point I'm trying to make is Anthropic's marketing about broad security risk related to the capability of its models is a valid concern though their dog and pony show really overdid it, probably to the detriment of us all for many reasons. It is indeed amplifying the abilities of people to find and exploit security issues.
The point of my anecdote is I was able to identify and fix an at least security adjacent bug in a language I could charitably consider myself a novice in. It happened to very unlikely have a security impact, but that was mere chance. LLMs expand the pool of people able to find and exploit security problems and we're all considerably more vulnerable as a result.
The biggest security threat was always someone bored with $20, a lot of attacks could be ignored or at least not prioritized with that threat model. This isn't true any more and our attack surface has gotten a whole lot larger.
Find it yourself. On any recently released erlang create an ssh server with their library. With the only available post-quantum algorithm connect to the server 1,000 times. You should get one or two key exchange failures (1/512 chance to fail)
OP described a simple 2-line fix that would have been annoying to find by hand. That's a matter of heuristic search. The majority of problems in software engineering do not fall in this category.
More than low hanging fruit, I think it would have been legitimately hard to find. It only triggered 1/512 runs and probably would have required some expertise in crypto algorithms.
BUT regardless, pruning low hanging fruit for any task IS a force multiplier. So much of so many tasks are easy but tedious. Finding libraries, documentation not matching code thus reading code, correct syntax/arguments, and just tons of straightforward tasks which are not HARD but time consuming.
Don't forget, its no longer cool to say that now that the public has pushed back. The fact they all changed their tone away from taking jobs tells you that it was all just entirely marketing.
But the propaganda deluge was a smash hit so far, HN is drowning in “AI” BS, and astroturfers and spin doctors haven’t seen that much business since the cold war. They made more profit than shovel salesmen in the gold rush.
The US has gone all in on AI because it is one of the few things in which they still have an advantage over Asian countries. I wouldn't use the word pathetic but rather "desperation".
So is your position that i.e. the Five Eyes [1] cyber security leaders are just pretending that AI cyber security is a serious thing to play into the geopolitical east vs. west thing and its not genuine?
It just feels like people are starting to reach for conspiracy theories rather than engage with the idea that these models might actually be dangerous.
The “Five Eyes cyber security leaders” aren’t exactly famous for their political independence, or for having the public’s best interests at heart, or erring on the side of regulating less.
You don’t get very far in the spying profession with honesty.
I'm just glad there are so many jobs. Just look at the latest unemployment numbers! I wonder if this era of peace and prosperity will be remembered as the peak of humanity?
And did you see that chocolate rations increased again last month! It's literally incredible.
If Mythos is still running internally, the NSA still have some access to it. It's just crazy to believe there aren't CIA and/or NSA plants (tacitly acknowledged or otherwise) inside Anthropic and OpenAI.
But Mythos is still only an advanced LLM so I am not sure what all this breathy fuss is about; it sounds like the PR war more than anything.
If the NSA aren't themselves training technologies that are at least as powerful, that would modestly surprise me.
Not that you need an LLM to monitor the risks to the USA. You just need Tulsi Gabbard's emails.
I think it’s beyond a mastery of PR. They literally called it Mythos and built a literal myth around it. I mean… maybe people just want the soap opera.
> That contract has not been finalized, and some Pentagon officials want the N.S.A. to find a way to work with other models.
Good, fsck NSA, that's the last organization I'd ever want to have access to Mythos. I hope this administration's incompetence will prevent them from regaining access for as long as possible
It’ll be the first organization to get access to Epic/Saga/Legend/Bible/Torah/Sutra/Vedan/whatever the Mythos+1 is called - and it might be the only one with this privilege
More likely they'll convince congress they will need their own. Only it will 20-200 times more expensive and the US taxpayer will be paying for it but won't get access.
That's is funniest thing I read since long time :)) I mean: it's so absurd, almost like things we had in real socialism in 80s :> But, yeah, freedom have consequences.
honest question: does the nsa have the abilty to take the model weights from anthropic? also, as I understand it anthropic employees from the USA have mythos access, and I dont see why this shouldnt extend to the nsa. this seems pretty silly and kinda unbelievable. commenting again to add more infomaiton to my opinion and ask that you don't just blindly downvote bc I don't believe the nsa doesn't have mythos
If they wanted to officially take the weights the DPA would work and Dario could do nothing. If they wanted to do it in clandestine manner no one could stop them and no one would know. It's very likely they already have all the weights from all the frontier models. I mean all the frontier models are capable of being served from AWS Bedrock so the weights aren't exactly locked in some air-gapped vault.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
Pretty sure even under DPA, taking without fair compensation would be a violation of the takings clause of 5th Amendment and wouldn't withstand legal scrutiny. If they wanted to get them clandestinely, yeah, they'd likely get away with it, but it is stealing.
Serious question: do you think the NSA aren't training their own LLMs? (With or without Anthropic and OpenAI's help)
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
I don't think there is much overlap between people capable of building cutting edge LLM's and the people who want to build a cutting edge LLM for the government.
The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
Mathematicians in academia are paid a little less than AI researchers. Companies are willing to pay billions to steal the few people capable of driving development of frontier LLMs from each other. Cryptographers don't quite enjoy the same popularity.
> The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
That sort of proves the opposite point, assuming you're referring to Dual EC DRBG, because the flaw was noticed very early on, by people who weren't even involved in its development.
> Serious question: do you think the NSA aren't training their own LLMs?
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
Serious question, do you realize that the NSA are mere mortals? Do you realize how much it takes to train a model? Does the NSA make their own chips or planes? The NSA buys a lot of technology because they can't make their own.
NSA has had their own supercomputing program for decades. they design and produce their own large scale machines. chips, fabrics, arithmetic units, all of it. they also employ quite a number of hardcore mathematicians, computer scientists, and systems wranglers. if they decided it was of strategic importance there is absolutely no reason they couldn't train their own models.
I guess we're just conspiracy theorists for landing at the objective conclusion that three letter government agencies:
- find "modern AI" to have strategic importance
- have ways to spend loads of money while having a front-facing budget on the record
- could be running a PR program to have Americans think they "buy" access to models like they do, but the AI companies were taken over by these agencies long ago
Look at Google, Microsoft...Apple got away with it by having as much on-device operation as possible so they could wash their hands, honestly saying "We don't have it."
This is the world's largest data gathering operation. Remember after 9/11 when the NSA copied as much Internet back bone traffic as they could?
I'm not for or against, even as a resident, but we certainly shouldn't be naive.
as someone who actually worked at the NSA pointed out earlier in this thread, they have plenty of resources, but also plenty of politics and some execution problems. so I wouldn't put money on them making a great model, but to say that they are completely incapable of doing anything is probably quite wrong.
the issue here that is a forgone conclusion, regardless of where the model comes from and which chips it runs on, is that now they can reasonably comb through all the stuff that they've been collecting. that's a pretty huge operational change.
You mean "Rhetorical question," and I didn't need patronising.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
I can't say what they're doing now because I worked for the NSA 15 years ago but the view of them as an omnipotent power is a product of Hollywood. The government is good at throwing an ungodly amount of resources at something to get a result through sheer attrition, and so they are often the source of original development of technologies. The private sector has always been much better at building a technology to greater sophistication and efficiency. There may be blue badgers in Fort Meade trying to train models but there is no chance they are competitive with the frontier AI companies. It's like saying the government has an amazing home-grown fighter aircraft that is beyond what Lockheed has ever made...they delegate that stuff to private companies for a reason.
Blue badges were for government employees (like I was), and green badges were for private contractors. And yes they have a lot of math and physics guys; my own physics lecturer was in my orientation class, actually. He was there for quantum computing, which reinforces my point. The government can be good at pioneering unproven / uncommercialized technologies, but in general they are like a blunt weapon; the profit motive and lack of bureaucracy eventually makes the private sector far better for improving the technology later. In the case of LLMs, they didn't even originate in government, and I don't think there's any chance they are being developed there at a more advanced level.
Crypto and AI are deeply connected, and you see similar structures/problems in both. Shannon, the “Father (or whatever) of AI”, worked for the NSA and published many papers there that were later declassified.
Here is a banger quote on this by Shannon’s boy Warren Weaver, keeping in mind LLMs came from translation problems:
“One naturally wonders if the problem of translation could conceivably be treated as a problem in cryptography. When I look at an article in Russian, I say: 'This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.”
The NSA is government agency. They are certainly not training any world class LLMs. They probably have some specialized fine tunings of existing models, but that's it. They don't have the capacity.
Seems to me OP's implication is that they were fired because someone wanted to hit a quota of (employees cut/payroll expenses reduced), or other similarly ''reasonless'' justifications.
You're assuming that they are "dismantling" it in a sense, what they are actually doing is mostly attacking workers while introducing extremely unsafe software. If you think LLMs are terrible, imagine it being the gatekeeper on whether your personal info is shared to an individual (and they ain't wasting time on the boring info either!).
When you want to reorient the government, it's much easier doing it with a smaller more loyal force. Now introduce tools that make mass surveillance easier and less accountable.
Like that's not a bad thing for them, that's what they want to do.
---
Back to the article, I'm not shocked that a massive LLM company speed running into the brick wall that is the US government; just thought it would be OpenAI, but Sam Altman is truly the best bottom feeder the game.
Also fully believe that Anthropic is hoping that public sentiment is on their side but more Americans hate AI companies than Trump so it's not going to go how they want.
Give it maybe 3-6 months before the Trump Admin talks about openly nationalizing Anthropic.
The NSA is managed by the NSA director, an independently appointed and confirmed office separate from the Office of the Director of National Intelligence. The DNI does have the authority to restrict funds to the NSA, and sets certain priorities, policy, etc. but the NSA director is not beholden to the DNI and makes their hiring and firing decisions independently. They’re also, currently and historically though not required by statute, a flag officer in the US military and dual-hatted as the commander of CYBERCOM. All this is to say, chaos in the office of the DNI does not necessarily impact the NSA.
Anecdata suggests NSA just got on board and kept going tbh. Not sure they’ve felt the same impacts / churn as other agencies, and not sure they’ve ever really been that beholden to the DNI.
I don't want a single person to be working at NSA and I find ANY terms to effectuate that outcome to be acceptable. Very much so including DOGE, which I despise.
Yes. But unlike the rest of us, NSA didn't have to if the administration had thought about it for 30 seconds before sending their letter. It's a stupid own-goal.
NSA has produced nothing. Does nothing. Why don't we just have them pick up garbage on the freeway to help out the tax payers? Let Anthropic and other adults push spying forward.
Have we become reddit here? I mean, you probably have some sympathy and upvotes from fellow readers, but this isn't pushing the conversation forward at all.
Any citations to your statement that NSA produces nothing? Or do you have a strong argument or evidence to support this?
You misunderstand - the government issued a directive to Anthropic that effectively forced them to pull access from everyone, even their own employees.
The directive was to remove access to non-Americans, not to pull access from everyone. It’s because Anthropic cannot verify the identity of its users that it pulled access from everyone, not because the government explicitely requested that.
If their operation team is not US based that's going to be difficult to operate. They would have to reorganize the whole company as I'm pretty sure that they are not employing only US citizen.
Well, kinda in effect. He lacked the authority to make the call, and it is quite obviously being ignored by most suppliers. If it were actually enforced, I believe no companies that are contractors for the DoD could even host Anthropic- like GCP, Azure, and AWS. Perhaps they are currently figuring out how to get off all cloud provider govclouds, but I doubt it.
Which is a fundamental mistake to make with the U.S. government, even if we’re talking only about the executive branch, even if we’re only talking about DoD, even if we’re only talking about the IC.
Yeah... NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company (based on my reading/following of Snowden leaks and others). Anthropic wouldn't be able to exist without implicit NSA approval. This article reads more like a marketing piece for Anthropic/Mythos... and ends by talking about how much NSA wants Anthropic models.
It's just not technically feasible, so there's nothing to lie about. They're not MITMing petabytes/sec across dozens (hundreds?) of companies and they haven't broken TLS1.3.
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
Of course it's feasible, you just intercept the traffic post-decryption on the cloud/server side. You don't control how/where your traffic to 3p cloud services is decrypted.
You keep saying this, but it's nonsensical. If I terminate TLS on the box that does processing, there's nothing to intercept.
And these days (especially post-Snowden), many (most?) companies encrypt data when sending between servers within their own (private network) infrastructure.
You have no control about where TLS is terminated when you're talking to a 3p cloud service (with services you don't control/run like cloud LLM APIs). You also have no control about what spyware is installed on/around VMs you rent (and there's a lot). Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point, not to mention every major US tech company generally cooperates with the NSA and gives them access to anything they request (including allowing the installation of dedicated hardware to intercept decrypted traffic as has been publicly exposed documented many times already).
Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement - if you actually looked into what happened post-Snowden - absolutely nothing was done to prevent NSA spying on any communications they want, in fact it got significantly worse.
> Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
lol, no, it's really not.
> Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point
Why would I want the data to be decrypted at each point and why would datacenters do that? Encrypting and decrypting data is expensive computationally, so that's not how things work at all. There's no need to decrypt data to know where it needs to go. That's why we have TCP/IP and other similar stadards.
The datacenters can maybe add another layer of encryption on top of my data as its moving around their networks, but there's absolutely no way for them to strip off my encryption.
> Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement
Things didn't magically get better. A lot of people worked hard to improve the overall security posture of the industry.
> Why would I want the data to be decrypted at each point and why would datacenters do that?
I think they mean the data must have existed in plain text before it was encrypted, and will exist in plain text after it is decrypted.
At some point “your” server in a datacenter somewhere needs to decrypt the data to do something useful with it, after all you’re paying for compute, and homeomorphic encryption is too slow, so the work is done in unencrypted data.
There it is. Your data in plain text in RAM.
TLS will protect your data in transit, but it can’t protect you against a compromised recipient.
So the NSA streams the ram of every virtual machine and bare metal server on the internet to themselves so they can analyze the plain text that's being processed in ram and no one has noticed this network traffic? How could that even be possible? If I buy a 100Mbps network connection from someone, they just provision a bit more so that the NSA streaming doesn't impact or show up?
> Why would I want the data to be decrypted at each point and why would datacenters do that?
When we talk about data that is sent for processing to a 3p server (like anthropic in this case) the data obviously needs to be decrypted to be processed.
As to why data is decrypted at each point in a typical large backend system - because other than network routing there are presumably multiple services that need to receive and act on this data somehow - you're not just sending encrypted data around to random servers.
> there's absolutely no way for them to strip off my encryption.
You don't seem to understand that you have no control over the encryption or decryption done on the backends of cloud services you use. I don't know how to make it more simple and obvious at this point.
Again, the context here is Anthropic and sending your data to their (or any other big tech API). But even if we move away from this model and suppose you are running your own services on rented cloud VM - then it should be obvious that you don't have full control or even access to this VM... any actor with access can install or modify any software, install/modify EBPF, modified crypto libraries, etc. - you have absolutely no control or say over this.
> Things didn't magically get better.
Things didn't get better at all, they got much worse.
You just intercept the traffic after its decrypted on the server side, or are you suggesting you somehow send encrypted traffic that never gets decrypted?
So the NSA streams the memory contents of every virtual machine and bare metal server on the internet to get the decrypted traffic? How would that even work at the scale of the internet?
How it works is they build a huge virtual strawman which decrypts and reads all of the data for them then posts online about how NSA spying on people is literally impossible.
I second this: HTTPS (as most consumers use it) is probably a front (who are these CA's really anyway?)
Plot twist: _Perhaps_ Mythos / Fable keeps explaining ways (that we can't comprehend or don't always work) to break HTTPS due to the three letter agencies making sure they had input on their creation (and thus backdoors, I mean "bugs"), so the real catastrophe they are hiding is that HTTPS is broken (for most people, most of the time.)
Remember when Quantum computing was the threat to HTTPS? Turns out it was the humans own inability to think outside of the box!
Which would be easily detectable if the cert I'm using on my server didn't match the one that was being served publicly.
There's really no way this conspiracy theory works if "they" have a copy of every single private cert generated. Which would be impressive because I can generate one myself and get it trusted without ever sending it and would be easily able to detect a MITM attack.
Not to mention most sites are going to use pinned certs so any repeat visitors to a site will notice a cert change associated with a MITM.
This whole idea relies on the assumption that everyone is trusting third parties with their private certs. That is not at all required.
> How are they going to MITM communications with certs that never left my machine?
The long game. They:
- make sure you wouldn't be in a position to need to transmit data anywhere that would receive it without CA's in their hypothetical pocket
- manage the evolution of the cloud industry to make sure portable VM's and Containers can have their data archived (both in-RAM, disk, hey just send us the running VM!)
- backdoor'd encryption algorithms from the design and implementation phase to ensure a global unlocking mechanism for any data encrypted by anybody who used a large class of extremely commonly available software
So, you run your own private bank in a cloud VM with tenant managed keys? They backdoor'd the encryption algorithm your cloud VM disk relies on, because they blackmailed one of the developers at the company who developed the hypervisor system used by your provider. Open source project? Perfect. (If you think this is nonsense, then remember the rapid discovery of ancient "bugs" causing all this drama to begin with.)
Your TLS privately generated certs that are 100% foolproof aren't actually used anywhere encrypting the data they want, because it's either worthless, or, available elsewhere perhaps at a different (or same) time.
If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data. This proves my point. This shows the effort required to infiltrate _one_ target and you're suggesting they've infiltrated everything by default.
How would you know about the successes? Thinking this is the one and only time they tried it is... interesting.
(Plus: "it was, for seven years, one of four CSPRNGs standardized in NIST SP 800-90A")
> If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data.
No, this demonstrates an actor of that power level doesn't even need to compromise encryption, and can get deeper access to everything, if it's worth it to them.
I recall having a nuclear meltdown personally when I heard about all of this in the mid aughts. Nobody cared. Nobody understands this today. Everyone just complains about the Donald, but I point to this, and they don't realize the connection.
It's generally accepted fact that the NSA broke HTTPS, for some of the time, for some of the services. It's unclear what they do have, but you'd be naive to assume consumer HTTPS is keeping them out.
It's too complicated. Do you know everything about CA, SSL, HTTPS, and so on? You make $250k a year working on it? Do you _really_, _really_, know everything? Then you're fired because you're lying to yourself, so you're probably unbearable to work with.
We were all freaking out about this with AT&T Thing nearly twenty years ago: and when nobody cared (Bush ran two terms! it helped to pretend AT&T was the only one affected), it gave "them" implicit permission to do it again with Google / Yahoo thing (it helped to pretend those were the only two cloud providers affected) ten years ago.
Now, we're all pretending that capitalism is real, and that the three letter agencies are just sittin' on the sidelines, while the world's largest data archiving opportunity is happening voluntarily (some are even PAYING for it!), at some wild-growth companies (with leaders who have too much to lose), who also have existed for just a few years? A 5 year old could probably blackmail Sam Altman, what about all the other middle management? The individual contributors (if they still exist) are of no concern: work is a commodity, it's easy to silo a worker's knowledge.
Surveillance opportunity is 10x social media from last decade, because they still have social media, and now, they've began thinking for people. How easy when it is an app on your smartphone. Those mind control experiments back in the 60's with Acid are looking silly by now. Besides, how do you know that the response you're getting wasn't manipulated (and define 'manipulated' across a spectrum of training to nefarious actors impersonating models, by power of court order.)
If you think all of that is unfounded ridiculous blasphemy, let me distract you with this instead: if the AI bubble bursts, the compute will be repurposed for mass AI / ML driven CCTV surveillance. Hell, maybe they'll find a way to give you a tax break if you sell your CCTV footage.
"NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company" even if this statement is an exaggeration, by playing the long game, they get themselves setup to access what they want in the future.
I'm not for or against, but I do live in a safe place thanks to such surveillance (generally in the USA), and I want you to know that this AI Thing is only the latest chapter in the intelligence story.
>they're cops with too much access, it doesn't take a genius to outsmart a cop
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
> every math phd i am acquainted with has been approached by nsa recruiters.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
>how many of them took them up on the offer, and how many are in leadership roles?
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
> this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
What? That's not only relevant to the point, it's incredibly relevant. If the NSA is only able to recruit 2% of the math PhDs they approach, then that's important information.
"More than one" is not particularly useful; you seem to be dodging the question because it undermines your argument.
telling you exactly how many people i know in the NSA is also not particularly useful. i'm one guy. there is no statistically significant information from my answer.
>you seem to be dodging the question because it undermines your argument.
my "argument" is that there are plenty of smart people in the NSA. that's it. i am confused why that is seemingly so offensive to you that you had to reply twice.
> how many of them took them up on the offer, and how many are in leadership roles?
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math phd market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
> it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I don’t think you have context on what math phds are making in entry level positions, post-docs, or adjuncting. I just picked a random entry level NSA role on LinkedIn (doctorate + 0 yrs) and they’re offering solid six digits. There are tenured faculty (post-doc(s) + 5ish yrs) who don’t make that.
No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
Perhaps you're exaggerating for effect, but that also undermines your point.
>No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
if you read my comment like we're having a normal conversation instead of a thesis defense, you'll get my point just fine.
I worked on these cases at EFF and I'm skeptical of the automatic "NSA has access to everything" intuition.
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
Propaganda indeed: my instinct says we are being lied to about how three letter agencies and military are paying for services. They give us a PR front that Uncle Sam is a regular paying customer just like you and me, but they're probably running the show: this is the largest data gathering operation since 9/11.
Sorry everyone: but the conspiracy is so obviously not, it's nauseating to admit, because you see all your friends, family and co workers dumping so much everyday data into these services.
The current position seems to be no-one has access, not even Anthropic employees. What powers does the US government have to force them to provide access? If they have that power why did they not use it to force them to provide their products for military use?
Probably not. The US constitution limits what government can force on the people. If the NSA tries to force something that will spend years in court (if anyone wants to fight)
I hear you but, the Patriot act was the gateway. View it as a spectrum from then and how the Administration is now, and suddenly what the Donald is doing doesn't even seem bad: it seems on par for the dystopian road-map laid out long ago (I can't speak for before 9/11)
> The US constitution limits what government can force on the people.
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
Kind of crazy actually. Other models are catching up fast, they all can find the vulnerabilities in our (and by our I mean everyone's) underlying infra very fast. It takes a very long time to fix, review, and finally deploy these fixes. There really isn't much time left.
I really seriously doubt this. no proof, obviously, but this sounds too unbelievable. the NSA probably has the mythos pretrain and might be finetuning it themselves.
> The White House and intelligence officials had pushed forward a classified contract between Anthropic and the N.S.A., which would allow the spy agency to use the company’s technology for a variety of purposes, including intelligence analysis and detecting new computer vulnerabilities.
Ironic that both sides are playing a horse shoe game:
Gov: The model is both a supply chain risk and also we'll DPA you if you don't give it to us.
Anthropic: The model is both like a nuclear weapon in terms of national security implications and safe for general release.
I mean graphite control rods do exist in nuclear reactors to absorb excess neutrons, preventing the fuel from going critical & making it technically safe for general use (THOUGH of course disasters have happened)
'Mythos “broke into almost all of our classified systems, not in weeks, but in hours.”'
Is Mythos a significant danger?
The curl experience does not suggest that hysteria is warranted, but this gives me pause.
Or, alternatively, it may suggest that the NSA’s classified systems are not very secure, which seems at least as possible: they may rely on requiring physical access to these systems to even attempt to penetrate them.
>> The curl experience does not suggest that hysteria is warranted, but this gives me pause.
What about the Firefox experience?
Or are we conveniently ignoring things that don't confirm conclusions we've already reached?
I'm not as familiar with that. I do agree that it sounded substantial.
I just think that a coreutils flaw is not as substantial as a rendering engine exploit.
Hadn't they spent a year hardening curl with various AI before they tried Mythos?
Yes. The original curl post didn't say anything like "mythos sucks" but rather "it's only a minor improvement in comparison to already widely used models".
Yes, and Firefox had not.
Which I think points at Mythos not being some big jump in capability finding things earlier LLMs didn't, it seems to mostly come down to massively increased compute budget and they finally catching up in context sizes.
Aren't you trying to do the same thing. Llm people, you're cooked.
Curl is such a small utility, and the effect of any single problem is limited.
Mythos's great strength was finding multiple vulnerabilities and chaining them together to break a whole system.
Look at it like this: It found one confirmed, minor vulnerability in Curl (but I don't think they have said what it was?). In another system that used Curl it's possible it could have exploited that vulnerability to chain to another, bigger vulnerability that was normally inaccessible.
That's how systems get broken.
Why are these things online at all? Is that a requirement for them to be useful?
'Mythos “broke into almost all of our classified systems, not in weeks, but in hours.”'
And the government's response was to limit access to US citizens? I don't believe this for a minute. If Mythos could actually break into all these systems, the government would declare it a national security risk and it would never see the light of day for anyone outside government staff with security clearance.
additional context from the article regarding that particular statement:
"[the statement] was oversimplified... In reality, the tests involved “red teams” of N.S.A. analysts who were using Mythos in a highly tailored environment that would be extremely unlikely for an adversary to replicate, officials said. The red teams began their tests within classified N.S.A. systems designed to be accessible only from certain computers and completely cut off from the broader internet.
The tests found that Mythos was able to identify cybersecurity flaws within that classified network quickly, but it did not actually break into those systems, the officials said."
mythos allowed mediocre people to get results by holding their hand through the process, or just ignoring their irrelevant input and knowing what to do.
if you throw millions of tokens at IDA Pro MCP with the right prompt lets just say security by obscurity fails miserably because there is no obscurity when the LLM chews through the decompilation.
It isn’t bad, it isn’t good. It’s just how the world looks now. All software is open source now, some of it is just more open, some of it is less.
>mythos allowed mediocre people to get results by holding their hand through the process
Isn't this what technology progress looks like? Industrial tools allowed mediocre people to improve their productivity by orders of magnitude which is how we managed(in the past) to build so many amazing things with less human toil and suffering than previous generations.
Progress isn't always welcome by the incumbent who have built their moats on hoarding knowledge over being adaptable
It seems like AI is really hurting the people who don't have a hoard of experience - the juniors and early mid-level tech people.
The incumbents with experience are doing amazing. PM's with Mythos aren't replacing the PE with 20 years of experiences lol.
I mean that is what most technology looked like at first too.
Oh okay. So where's the point where AI starts to encourage the development of a new useful skill set among people early in their careers?
Not all of us think encouraging people to outsource their own thinking to proprietary models is actually "progress."
Are you saying programmers aren't adaptable? I don't think I've ever seen this field pause to take a breath.
> mythos allowed mediocre people to get results by holding their hand through the process,
Yes, just like early cars allowed mediocre horse riders to get from A to B with dignity.
Or like my Japanese rice cooker allows a person like me, utterly shitty at preparing this, to eat some rice that is cooked to perfection.
Etc.
I mean, the calculator is my go to analogy I keep bringing up in this debate.
It lets someone with mediocre long division skills to just do the thing they need to do with fewer steps and less friction.
IDA itself is a tool that helps you decompile code without having to do a lot of things.
knowing long division does not help you make the calculator do division better.
Understanding math absolutely makes a calculator more useful to you though.
and if you work that into the parent's analogy you get the point I was making
Apologies. I misread the comment to which you replied and gave them unwarranted credit for not making the same tired point about calculators.
Should mediocre people be preforming heart surgery?
Lots of mediocre people already are.
It depends. Mediocre doctor in a remote area with right tooling assistance as opposed to no one being available for someone who urgently needs one? Yeah this should be a thing. Should a software bro in NY perform it in dark alley despite having best doctors few blocks away? Maybe not…
I'm sure many 'mediocre' people perform heart surgery. Only 100 years ago, the idea of a person without a certain surname or race, would've been a ghastly preposition, no?
Do you... think heart surgery has become LESS dependent on surgical skill in the last 100 years? Cardiovascular surgeons spend MORE time in training now than they did 100 years ago.
Did heart surgery as we know it exist 100 years ago, or are you trying to conflate things to make a point?
"heart surgery" isn't a technique". Name something, literally anything connected to the profession, and tell me whether the training time is naturally bound to keep going up and up.
"mediocre people"
I'm glad to see the mask is falling off the privileged caste.
Is there anything inherently wrong about open access to tools? (Apart from rent payments).
The "privileged caste" being people who actually expended the effort to learn things for themselves?
And such people learnt everything from the beginning? From fire?
Where's the cut off point of where learning something for yourself becomes the signal for entrance to the enlightened caste?
It's the point where you expend effort to learn a useful skill.
> (Apart from rent payments).
Privilege enables you to rent competence, historically by paying other people. The slop companies will now sell you a simulacrum of competence by the token.
The fact that competence can (could?) only be acquired through sustained effort over a long period of time is (was?) levelling the field.
Selling simulated competence perpetuates privilege, instead of dismantling it like you seem to claim.
Is there a historical precedent as to what happened when the upstart denied capability to the empire?
The closest I can think of is the bronze age collapse.
There is no consensus on what caused the bronze age collapse.
Thenn, it makes it more riveting when modern day phenonema happen, surely?
Unless you subscribe to a historical channel?
Sorry i forgot to ask. What are the top 5 theories, and do you see any modern parallels?
Are you an AI?
Try not to be. Apoologies if the link aint obvious enough.
Perhaps, but I think volcanic eruption followed by system collapse is very compelling. Here is the story I find most convincing from the experts whose works I have read.
It likely started with a volcanic eruption, leading to widespread famine. Those in western Europe who didn't want to starve migrated en masse, as whole families, becoming the sea peoples. The powerful empires struggled to feed their people, and many were destroyed by the forced migration from the sea peoples. Egypt barely survived, but only as a shadow of itself. Many of the others were destroyed by those who had survived on marginal lands and didn't need complex societies to keep themselves fed.
Iron can't be the cause, as iron weapons pre-existed the Bronze Age collapse. I think the evidence is stronger that the collapse forced widespread adoption. The collapse devastated long-distance trade networks, which cut off the supplies of tin needed to make bronze. The scarcity pushed people to rapidly improve iron smelting.
I'm not a professional historian, but I do find the topic interesting. We should try to learn from past disasters to prevent repetition.
See Eric H. Cline's "1177 B.C.: The Year Civilization Collapsed";
Epimethius video "What was life like after the bronze age collapse (extended version)" https://www.youtube.com/watch?v=uM6JSS3l-IQ
What's the time period between iron being widespread and the so-called collapse?
> The closest I can think of is the bronze age collapse.
No idea about your question, but I'd love to hear more about this part.
AI marketing bullshit stunts are unlike anything I've seen in 30 years. It started with MS Copilot so called capabilities for work, which were completely made up use cases that didn't work at all (3 years later still). We've had OpenAI "AGI is coming" and "AI will take your job", now we have Mythos being so "dangerous" for cybersecurity, which of course makes the average Joe interpret it as Anthropic being "the better overall company, the NSA uses it!!". I mean gov foes with Anthropic are probably true, but the marketing is to blame not Mythos capabilities. This is all so fucking pathetic
All for a product that has yet to make a single honest dollar in profit for anyone who isn't nvidia.
When this goes we might well see a recession. Not that anyone responsible will be worse off, of course.
The perpetrators all have their golden parachutes. The taxpayers will foot the bill.
The US is trillions in debt. We live in the age of magic- nobody foots the bill.
Why on earth would you expect any of them to take profit so early in the game?
Silly me, expecting a company worth a trillion dollars to make... some money. Any money. A single profitable product.
Well the good thing is you’ve done the homework to definitively demonstrate that this is remotely true. These confident claims of this all being some sort of unprofitable Ponzi scheme, not understanding the concept of a growth phase which a multitude of highly successful tech companies have already demonstrated work while simultaneously commenting on a site with YCombinator in the url are just getting amusing now.
Of course this is a profitable technology, and it doesn’t matter if any of the labs are profitable today or not. Running at a loss is a perfectly rational strategy.
Anthropic is running at an operating profit this quarter: https://www.wsj.com/tech/ai/mind-blowing-growth-is-about-to-...
(It's actually probably more profitable than their projections here calculated because they were expecting to be running Fable but can't, and Opus costs less to run)
We should seriously reframe this whole AI thing to "SI = simulated intelligence".
It's google in a box. Great achievement, makes knowledge work faster, but please stop bothering everyone else.
The Uber and Groupon people became billionaires, so the "Simulated Intelligence" folks will also achieve it. No need to worry and drown everyone in these bs stories only non-tech people believe.
Heh. In the Schlock Mercenary universe, "SI" means "synthetic intelligence", which is a level below real AI (which means what we would call AGI). And, as it says (in https://www.schlockmercenary.com/2003-07-21), SI translates to "kinda stupid".
Can you describe your experience using modern AI tools that led to this conclusion? It is hard for me to wrap my head around how my perception could be so different from someone else in presumably the same or similar profession. I'm not asking this in bad faith either but I think your getting downvoted because your comment comes off as a pretty strong assertion without giving details on how you got there.
A lot of effort is spent to make the "conversation" feel just like a human-to-human interaction. This is not a naturally occurring phenomenon due to the technology, but rather a feature carefully engineered by those companies in order to get people hooked. Then they have all these tiny nudges like the typing animations or the "thinking..." popups before the next chat message appears.
At some point you might have also noticed the over-use of emojis, the bolted-on jokes, and the tendency to always approve what the user says (even though they have toned that down after backslash). At some point too many people thought they were in a relationship with the chatbot, because it always encouraged and approved them, so they had to hotfix it.
It's a bunch of really dark psychological patterns that are carefully combined by very clever people in order to create the false illusion that the user is experiencing something deeper than an engineered simulation of human interaction.
I think the technology is really useful, but they are obviously not happy with simply replacing a google-like query interface, they want users to fall in love with the product and mentally treat it like a fellow human being - and that's what I think is insincere.
>A lot of effort is spent to make the "conversation" feel just like a human-to-human interaction.
We'll in humans we call this an education and it takes quite a long time to get one.
Not a good comparison. Education is the part where they train on all digital content they can get their hands at, no matter the copyrights.
You get your education, you can replace google as a query-response interface to all digital content.
But then they use system prompts to simulate a fake persona and a user interface such as female voices or chat conversation in order to suggest that one is interacting with a real human being. This is clearly aimed at exploiting vulnerable cohorts of people, because the knowledge base part of this innovative technology is already solved.
Like casinos and social media companies, they know the profit is in the "whales" who can be psychologically manipulated to spend their time and money against their own interests.
This is oddly conspiracy theory oriented.
How would you program a LLM so it gives useful information to people with the least amount of people bitching about it?
At the end of the day the LLM does not have a native persona. It has countless numbers of them. It can act like an autistic man, a flirty woman, a kid from some country you've never heard of. Bringing forth an agreeable persona from the myriad is a bad thing?
To get more concrete are you using coding agents like Claude Code/ Codex/ opencode etc? What kind of work are you doing specifically?
If you are doing the kind of median enterprise tech work these tools are just good enough to do it at a relatively high level or atleast heavily augment people doing it.
Examples would be like adding routine CRUD features to APIs/ improving observability/ adding tests or accessibility features to codebases etc.
It try to explain it better in my longer sibling comment. I'm not using any coding agents. Their engineers can't be bothered to design their own webapp properly so I don't trust their binaries.
For me both Claude and ChatGPT are query-response services and replacements for google. They help with error messages, single-file MVPs, and software design problems such as comparison of different modules.
In my experience everything that goes beyond 200 lines creates issues down the line, so I try to keep interactions really short. Of course they can convincingly add CRUD functionality or tests, but one needs to double check their correctness, and if the subtle bugs are finally spotted then one needs to fix them anyways.
It's good for a first draft but I wouldn't use agents on a codebase I actually care about.
Unfortunately the billion-dollar funding forces the AI startups to make a return, and they are finding it in a vulnerable cohort of people who respond positively to a simulated human interaction, which is why they are focusing so much on it.
The query-response knowledge interface was the moat of google, and nowadays it can be 80% replaced with a local GPU and an open model. They know it, which is why they try to hook people on the simulated human interaction aspect of their interfaces through chatbots and voice chat.
POV OpenAI, early 2021. You have a pretty good next-token predictor called GPT-3. You noticed sometimes it can do useful things if you write out the start of a task and let it predict the next step. However, sometimes it's very difficult to frame a task that way. So instead you train it to predict the answer based on a question or instructions. Oh wait, it didn't get it right the first time... better let users iterate too. Now you have a conversation.
Things like loading indicators are basic good UI dating back to the 90s.
A/B testing and generally following user preferences might still push towards the dynamic you're describing, as it did with gpt-4o. xAI and a few other companies like Replika also intentionally created "companion"/porn AIs. But in general, natural language was previously exclusive to humans. It's completely natural that the first technology capable of it would therefore be perceived as more human. It's worth trying to resist this tendency, but it doesn't require evil intent on the part of the creators.
The "conversation" interface is exactly the same workflow as one used with google search: user states a query, page loads, user adapts query because they are not happy with the result, until they end up with a suitable result which makes them close the tab.
So they have made this amazing query-response system which is far superior to google due to the summarization of query results from the global web and the auto-translation to present them in the user's native language. This is the type of raw query-response capability which many software engineers are trying to use in their agentic coding sessions.
However, after achieving such innovation, the AI startups consciously choose to apply social media KPIs to their query-response startup, which incentivizes all the dark patterns we have seen in their user interface. They notice that a certain subset of users can be tricked into believing that the startup's query-response interface has human-like qualities such as a name and persona.
This user cohort shows amazing metrics in terms of time spent on app, so they adapt their user interface and their system prompts accordingly. The AI startup doesn't have to care if the reason for humans accepting the illusion of a simulated human interaction is due to social circumstances (lack of emotional intimacy) or an underlying psychological vulnerability that the startup is actively exploiting.
The AI startup only cares if their "simulated human interaction" product receives negative attention from normal people who are not part of the vulnerable cohort, e.g. the suicides or the parasocial romantic relationships with the chatbots.
It is exactly the same as in the gambling industry: There is a certain subset of users called "whales" who are the cash cows for casinos, but if you look at the actual humans who are labeled with this term one can see pathological gamblers, most of which are ruining their lives and families. Casinos do everything to prevent people from jumping from their roofs after they lost all their money.
If AI startups can use simulated human interactions to make vulnerable people act against their own interests in the same way as casinos and social media companies do, it will allow them to make shitloads of money.
But if you're actually a clever person then be honest to yourself and others about what you are working on, and why these human-like features are really added to the user interfaces of OpenAI or Anthropic or the other AI startups.
So this is my framing of the situation.
I don't think this kind of problem can be overlooked by the insiders, and we might see some internal rifts along these lines: Will our AI startup simulate a human interaction in order to exploit our vulnerable peers, or will our AI startup focus on delivering the best response to our user's queries?
Because now we have local models, which - assuming one has suitable hardware - provide 80% of the utility in terms of a query-response knowledge base.
As we are currently seeing, the AI startups with billion-dollar funding have very big economic incentives to focus on the "simulated human interaction" part of the equation, because their investors need returns.
The biggest strategic blunder I see at Google. Because if Google actually changes their excellent query-response user interface to a chat conversation which simulates a human interaction with persona, name, and voice, then they knowingly pivot to the same social media KPI driven business as OpenAI and Anthropic are struggling with.
Feels like OpenAI and Anthropic are both more interested in B2B. OpenAI I'm more suspicious of after GPT-4o. I mostly use Claude and I haven't noticed anything that feels like an intentional dark pattern. It's constantly reminding me that it's a chatbot.
I was able to identify, diagnose, fix, and upstream a minor bug in and erlang/OTP ssh key implementation with Opus in maybe 20 minutes (+2 weeks or so for upstream). It is not impossible that I could have done this before, but it would have taken days or weeks. The actual fix was about 2 lines of code, hardly AI slop, but getting there would have been quite the slog, and I never would have done it.
There is a lot of the reason for AI skepticism out there, but people tend to do massive overcorrections and underestimate the force multiplier it can be, particularly for people with some idea of what they're doing and a good grasp of how to take advantage of the tool.
I said absolutely nothing about LLMs, which is a fantastic tool I'm using every day. I'm talking about marketing.
So let’s say you’re in Anthropic’s shoes. You see that LLM’s are getting better and better, and it’s very possible that they will have some impact on jobs in the next few years, and a very meaningful impact on cybersecurity.
Is it more ethical to stay silent about these concerns, as you might have a bit of self interest? Or even if it looks a bit self interested, is it better to warn people ahead of time? I think the latter is obviously the better position.
The issue is both OpenAI and Anthropic have lied so many times that it’s no longer rational to take anything they say at face value.
Also: they don’t have to know they’re lying to say things that aren’t true. There is definitely some cult-like behaviour at the moment on the west coast
Be specific, what do you consider their lies to be? Also, this is pretty straightforward. You have a decade of extremely stable and predictable performance trajectory. It’s easy to see the writing on the wall. You can feel whatever which way about their motivations and ethics but if you read say Dario’s raw words they are pretty reasonable. We have to have a good regulatory framework and do what we can to prepare ourselves while also not ceding a critical strategic advantage. The west coast is always cult like, that’s not new. And it ignores the very real substance to the discussion.
Every year since 2023 the models are too dangerous to release and in 12 months all white collar jobs will be obsolete. This might not have been a deliberate lie but it's clearly been untrue and they've said it again and again.
Predictions with wrong timing are frankly worthless. I predict at some point in the future the S&P 500 will be at 10,000. Of course I'm guaranteed to be right. But have I really predicted anything useful?
If Dario was really worried about protecting the sheep, he wouldn't cry wolf every five minutes because everyone knows that's the worst possible thing to do.
And if you want to ask if Altman is trustworthy... ask Satya Nadella or anyone else who's ever made the mistake of doing business with him
> Every year since 2023 the models are too dangerous to release and in 12 months all white collar jobs will be obsolete. This might not have been a deliberate lie but it's clearly been untrue and they've said it again and again.
How is a prediction a lie? Did they tell you "this will definitely happen in X time"? Their speculation is not only valuable (they are the closest to the technology) but also necessary (they need to buy long term compute contracts so these predictions are literally what they have to bet their real money and company success on).
They have said again and again that this will make an incredible amount of tasks obsolete, and they are of course right about this. The models _are_ dangerous to release, every time we hit the frontier. This has become _increasingly true_.
> Predictions with wrong timing are frankly worthless.
Who cares?
> I predict at some point in the future the S&P 500 will be at 10,000. Of course I'm guaranteed to be right. But have I really predicted anything useful?
You aren't cherry picking and strawmanning here? Should we have a tour of all of the things that have indeed been predicted well and already come to fruition? Was 2025 "the year of agents"? It very much was, wasn't it? Additionally, unlike the S&P, performance trajectory, for almost a decade, is incredibly stable and predictable. It's hard to know, a priori for a given task or category of tasks, what specific error rate will trigger a phase transition but it's absolutely obvious and clear that this will happen quickly. It has indeed happened quickly. Does 2026 coding look anything remotely like 2024?
> If Dario was really worried about protecting the sheep, he wouldn't cry wolf every five minutes because everyone knows that's the worst possible thing to do.
No you're right he would make well reasoned arguments for the types of problems we need to address urgently. Hmm...that feels pretty ethical.
> If Dario was really worried about protecting the sheep, he wouldn't cry wolf every five minutes because everyone knows that's the worst possible thing to do.
I don't feel either of them are trustworthy, they are CEOs acting in their companies best interest. But people suggesting Mythos delay was some sort of PR ploy is some of the most extreme mental gymnastics I've seen. I listen to the actual words spoken by these people and consider the hard data that is in abundance at this point. I listen to the large body of research on alignment and safety and measurement that anyone can read for themselves or use AI agents to digest for them.
I’ve just watched enough old Adam Curtis documentaries to know historically how these things always end, true believers in many dead ends have exactly this kind of zeal.
Very smart people, reasoned arguments, “science”, all wrong.
But maybe this time will be different
I think that Anthropic is fully absolutely unethical. And they lied a lot. They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.
If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
It really does not matter how much they believed own doom predictions, because they were actively trying to make them true whether realistic or not.
> They were actively trying to make the doom happen while trying to cash out maximally on doom trolling.
These words make no sense. Anthropic delayed mythos/fable rollout. A mythos model without safeguards would have been a pretty bad idea, and they sacrificed a ton of revenue and risked being scooped by any of the other labs in the meantime. Frontier models are only frontier temporarily until the next lab releases their model. Of course they are a company and need to act in their own best interest.
It is also clearly serious the problems we need to think about as we march quickly towards even more capable systems. Why on earth is it a problem to point this out?
> If they were actually concerned over social impact, they would try to minimize it. They could have sell their product as a tool to be used to make economy boom, they tried to sell it on promiss to make it shrink for most people.
What a really weird take; they employ some of the best safety and alignment teams in the industry and this is an active area of research that they are campaigning for more attention on. You complain about them “doom trolling” and then complain they don’t do anything about…the doom? No sense at all.
It is perfectly consistent to (1) sound an alarm and (2) March full steam ahead as quickly as they can. If they don’t do (1) that’s unethical. If they don’t do (2) someone else will. I would rather someone like Dario align these models than the CCC. Plus it would be nice not to have a war over Taiwan which is inevitable if China gains enough of the upper hand in this AI race.
Economic growth and short-term job loss are both results of automation. Anthropic seems to have been pretty honest about that to me?
If only they wrote in normal calm economic terms as you seem to imply ... and I wrote "shrinking economy for most people" not growing.
Was it more ethical for the boy who cried wolf to have cried wolf so many times that nobody believed him when a wolf finally did show up?
Be specific, what are you talking about. Industry has been continuously warning about many of the complex problems that are going to happen as a clear consequence of the technology. I don’t know of any problem they have talked about that hasn’t either already come to fruition in one sense or another or that just hasn’t yet arrived. Dario has been predicting the end of coding for a long time now and look where we already are.
So yea no it’s more like it’s important for industry leaders and those closest to model development to proactively identify the issues that they don’t have complete control over or that we don’t have a regulatory framework for.
Super puzzling to see these comments and of course with zero specifics just “they’re all liars and grifters”
I'm talking about the breathless alarmism that Dario and his company push out as a marketing strategy. They've given us such gems as these:
- "It’s a bit like selling nuclear weapons to North Korea" (from the company that can't go more than a day or two without serious downtime)
- "We are releasing a model that is too powerful for the public"
- "It would be good for the world to have the option to slow or temporarily pause frontier AI development."
- "I believe that biological risks may soon follow, and that serious AI autonomy risks may not be far behind."
You can fill my ear with nitpicks about there still being time for these cries of wolf to be born out, but be prepared for me to wax philosophical about all things being possible given an eternal timescale.
> Dario has been predicting the end of coding for a long time now and look where we already are.
Where? It seems exceedingly unlikely that developers have all been phased out while I wasn't looking, as Dario prognosticated. And even if they all up and disappeared, AI still hasn't found a toehold outside of the relatively niche market of agentic coding.
Are we really saying that Anthropic claiming AI would take over industries was some benevolent ethical move rather than marketing their product as a cheap replacement for human labor that works in any industry? Wouldn't the ethical thing, if they were actually concerned about labor displacement, be to shut down the lab and work to disrupt and disable other labs instead?
Oppenheimer believed that technological progress is inevitable: if something can be built it will be.
Anthropic (and Deepmind, and some at OpenAI) believe the same thing.
Their ethical argument is:
1) This technology is coming whether or not our company does it or not.
2) Strong AI needs to be under human control, and we are the best placed to develop techniques to make this happen.
To be very clear: Anthropic (at least) is very happy to restrict access to their best models. They have continually campaigned for regulation to make sure others have to do the same.
> Wouldn't the ethical thing, if they were actually concerned about labor displacement, be to shut down the lab and work to disrupt and disable other labs instead
Personally I strongly reject the idea that labor displacement is unethical.
It will be a serious problem to deal with, but that doesn't make it unethical.
The steam engine displaced labor. That doesn't make it unethical.
> Personally I strongly reject the idea that labor displacement is unethical.
Oh, well if you STRONGLY reject it I guess that's it.
> It will be a serious problem to deal with, but that doesn't make it unethical.
What WOULD make it unethical?
> The steam engine displaced labor. That doesn't make it unethical.
The steam engine also created new jobs to replace what it eliminated. It wasn't a mostly one-sided wealth transfer to the elite.
> The steam engine also created new jobs to replace what it eliminated. It wasn't a mostly one-sided wealth transfer to the elite.
Indeed.
You make my point for me.
What are those to be created jobs going to be doing that AI won't be able to?
There's two big differences with the steam machine: this change is happening much faster so society has much less time to adapt, and it's got a much wider scope. Steam machines only replaced a small category of jobs.
The point I'm trying to make is Anthropic's marketing about broad security risk related to the capability of its models is a valid concern though their dog and pony show really overdid it, probably to the detriment of us all for many reasons. It is indeed amplifying the abilities of people to find and exploit security issues.
The point of my anecdote is I was able to identify and fix an at least security adjacent bug in a language I could charitably consider myself a novice in. It happened to very unlikely have a security impact, but that was mere chance. LLMs expand the pool of people able to find and exploit security problems and we're all considerably more vulnerable as a result.
The biggest security threat was always someone bored with $20, a lot of attacks could be ignored or at least not prioritized with that threat model. This isn't true any more and our attack surface has gotten a whole lot larger.
What was the dog and pony show?
White House and Anthropic hold 'productive' meeting amid fears over Mythos model https://www.reuters.com/world/anthropic-ceo-dario-amodei-arr...
This and other things around April
> I was able to identify, diagnose, fix, ...
a link to the PR or Changelog would strengthen this comment that it actually happened?
Find it yourself. On any recently released erlang create an ssh server with their library. With the only available post-quantum algorithm connect to the server 1,000 times. You should get one or two key exchange failures (1/512 chance to fail)
Force multiplier? Or low-hanging fruit pruner?
What is the difference when every problem becomes low-hanging fruit?
OP described a simple 2-line fix that would have been annoying to find by hand. That's a matter of heuristic search. The majority of problems in software engineering do not fall in this category.
More than low hanging fruit, I think it would have been legitimately hard to find. It only triggered 1/512 runs and probably would have required some expertise in crypto algorithms.
BUT regardless, pruning low hanging fruit for any task IS a force multiplier. So much of so many tasks are easy but tedious. Finding libraries, documentation not matching code thus reading code, correct syntax/arguments, and just tons of straightforward tasks which are not HARD but time consuming.
> and "AI will take your job"
Don't forget, its no longer cool to say that now that the public has pushed back. The fact they all changed their tone away from taking jobs tells you that it was all just entirely marketing.
Seems to me that they were mostly right, and the message was received by the right people. No need to ensure it gets distributed to the wrong people.
All the CEOs very quickly changed their messaging after Altman's house got molotoved.
But the propaganda deluge was a smash hit so far, HN is drowning in “AI” BS, and astroturfers and spin doctors haven’t seen that much business since the cold war. They made more profit than shovel salesmen in the gold rush.
The US has gone all in on AI because it is one of the few things in which they still have an advantage over Asian countries. I wouldn't use the word pathetic but rather "desperation".
So is your position that i.e. the Five Eyes [1] cyber security leaders are just pretending that AI cyber security is a serious thing to play into the geopolitical east vs. west thing and its not genuine?
It just feels like people are starting to reach for conspiracy theories rather than engage with the idea that these models might actually be dangerous.
[1]. https://thehill.com/policy/technology/5936339-ai-cybersecuri...
The “Five Eyes cyber security leaders” aren’t exactly famous for their political independence, or for having the public’s best interests at heart, or erring on the side of regulating less.
You don’t get very far in the spying profession with honesty.
I haven’t heard anything about AGI in a long while. Oh yeah, and per conversations last Jan we were all supposed to be out of our jobs by now.
I'm just glad there are so many jobs. Just look at the latest unemployment numbers! I wonder if this era of peace and prosperity will be remembered as the peak of humanity?
And did you see that chocolate rations increased again last month! It's literally incredible.
If Mythos is still running internally, the NSA still have some access to it. It's just crazy to believe there aren't CIA and/or NSA plants (tacitly acknowledged or otherwise) inside Anthropic and OpenAI.
But Mythos is still only an advanced LLM so I am not sure what all this breathy fuss is about; it sounds like the PR war more than anything.
If the NSA aren't themselves training technologies that are at least as powerful, that would modestly surprise me.
Not that you need an LLM to monitor the risks to the USA. You just need Tulsi Gabbard's emails.
I think it’s beyond a mastery of PR. They literally called it Mythos and built a literal myth around it. I mean… maybe people just want the soap opera.
> That contract has not been finalized, and some Pentagon officials want the N.S.A. to find a way to work with other models.
Good, fsck NSA, that's the last organization I'd ever want to have access to Mythos. I hope this administration's incompetence will prevent them from regaining access for as long as possible
It’ll be the first organization to get access to Epic/Saga/Legend/Bible/Torah/Sutra/Vedan/whatever the Mythos+1 is called - and it might be the only one with this privilege
They will never be able to read all the words in my head that spell out exactly what I want to have happen at that org.
More likely they'll convince congress they will need their own. Only it will 20-200 times more expensive and the US taxpayer will be paying for it but won't get access.
That would meet the OP's goal of NSA never getting a frontier model, "behind schedule" is the natural partner of "over budget".
> NSA lost access to Mythos
That's is funniest thing I read since long time :)) I mean: it's so absurd, almost like things we had in real socialism in 80s :> But, yeah, freedom have consequences.
honest question: does the nsa have the abilty to take the model weights from anthropic? also, as I understand it anthropic employees from the USA have mythos access, and I dont see why this shouldnt extend to the nsa. this seems pretty silly and kinda unbelievable. commenting again to add more infomaiton to my opinion and ask that you don't just blindly downvote bc I don't believe the nsa doesn't have mythos
They could easily take the weights if they wanted. I don't believe they meaningfully lost access.
If they use the defence production act, would Dario be even able to resign in protest?
If they wanted to officially take the weights the DPA would work and Dario could do nothing. If they wanted to do it in clandestine manner no one could stop them and no one would know. It's very likely they already have all the weights from all the frontier models. I mean all the frontier models are capable of being served from AWS Bedrock so the weights aren't exactly locked in some air-gapped vault.
It would be easy to make a national security justification to take the weights in a clandestine manner especially because Anthropic supposedly got caught giving China access to the model through a cutout.
Pretty sure even under DPA, taking without fair compensation would be a violation of the takings clause of 5th Amendment and wouldn't withstand legal scrutiny. If they wanted to get them clandestinely, yeah, they'd likely get away with it, but it is stealing.
To be a taking, it would have to be property. Weights are almost certainly not property.
That's for the courts to decide.
Correct. What makes you think existing case law doesn't apply to model weights?
John Cook resigned, so Dario might resign also. But he would make it public, so they won't do it
> John Cook resigned
John Cook?
He means John Apple I think.
I think you mean Tim Mac
Oops, Tim Cook. Sorry
Who will make them the next set of weights?
If a government can just seize the product of someone else's labour, either they will end up as slave owners or without willing workers.
Serious question: do you think the NSA aren't training their own LLMs? (With or without Anthropic and OpenAI's help)
It's a perfect technology for their uses, they get a big chunk of a $100 billion black budget, and they've had access to the research for at least as long as we have.
I don't think there is much overlap between people capable of building cutting edge LLM's and the people who want to build a cutting edge LLM for the government.
The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
I wouldn't count them out.
Mathematicians in academia are paid a little less than AI researchers. Companies are willing to pay billions to steal the few people capable of driving development of frontier LLMs from each other. Cryptographers don't quite enjoy the same popularity.
Does getting paid more make people smarter?
Especially academia tend to do their work out of interest, their monetary gain isn't their primary goal
When people with a particular aptitude and skillset can make 10x as much money doing job A than job B, there is a bias towards job A.
Of course, that doesn’t mean nobody will do job B for other, non-financial reasons.
> The NSA managed to deliberately insert a backdoor into elliptic-curve cryptography right under the noses of everyone capable of making elliptic-curve cryptography.
That sort of proves the opposite point, assuming you're referring to Dual EC DRBG, because the flaw was noticed very early on, by people who weren't even involved in its development.
They probably also have an insane dataset
> Serious question: do you think the NSA aren't training their own LLMs?
Given the evergreen discussion of "are these companies making a profit"*, I think any LLMs that the NSA (or any other government agency worldwide) may be making are quite far from the leading edge.
* Person A: "they are making a loss!" Person B: "Only if you count training, they make a profit on inference, look at what it costs to run comparable open models on generic cloud servers" A: "Sure, but if they don't train new models they'll be left behind, so they're still making a loss"
That and the way compute is now measured in GW, I think even random low budget vloggers just getting started would be able to spot if the NSA was doing anything significant just from the extra heat emissions or power plants getting built.
Model training does NOT dominate the model costs.
The rate of inference compute to training compute is ~10:1, for popular frontier models. Models are routinely overtrained past the Chinchilla optimum now because it makes an immense amount of economic sense to do so.
Worse the more niche and unused your models get, but when this "making a loss" fuckery pops up, it's usually about the big guys like Anthropic, OpenAI, GDM and maybe xAI and Meta. Of which only the latter can be accused of not selling enough inference to offset the training runs.
The real money sinks are: R&D and infrastructure buildouts.
> do you think the NSA aren't training their own LLMs?
They probably already have access to Sentinel, so they wouldn't need to train their own.
You cannot really hide the amount of compute required to train an LLM. Do we have actual clues that NASA is training their own frontier model?
Serious question, do you realize that the NSA are mere mortals? Do you realize how much it takes to train a model? Does the NSA make their own chips or planes? The NSA buys a lot of technology because they can't make their own.
NSA has had their own supercomputing program for decades. they design and produce their own large scale machines. chips, fabrics, arithmetic units, all of it. they also employ quite a number of hardcore mathematicians, computer scientists, and systems wranglers. if they decided it was of strategic importance there is absolutely no reason they couldn't train their own models.
I guess we're just conspiracy theorists for landing at the objective conclusion that three letter government agencies:
- find "modern AI" to have strategic importance
- have ways to spend loads of money while having a front-facing budget on the record
- could be running a PR program to have Americans think they "buy" access to models like they do, but the AI companies were taken over by these agencies long ago
Look at Google, Microsoft...Apple got away with it by having as much on-device operation as possible so they could wash their hands, honestly saying "We don't have it."
This is the world's largest data gathering operation. Remember after 9/11 when the NSA copied as much Internet back bone traffic as they could?
I'm not for or against, even as a resident, but we certainly shouldn't be naive.
as someone who actually worked at the NSA pointed out earlier in this thread, they have plenty of resources, but also plenty of politics and some execution problems. so I wouldn't put money on them making a great model, but to say that they are completely incapable of doing anything is probably quite wrong.
the issue here that is a forgone conclusion, regardless of where the model comes from and which chips it runs on, is that now they can reasonably comb through all the stuff that they've been collecting. that's a pretty huge operational change.
You mean "Rhetorical question," and I didn't need patronising.
They have at least one pretty vast, largely classified data centre in Utah, with a sizeable chunk of the black budget and they also have pretty large data sets.
Whats in Utah is data storage.
I can't say what they're doing now because I worked for the NSA 15 years ago but the view of them as an omnipotent power is a product of Hollywood. The government is good at throwing an ungodly amount of resources at something to get a result through sheer attrition, and so they are often the source of original development of technologies. The private sector has always been much better at building a technology to greater sophistication and efficiency. There may be blue badgers in Fort Meade trying to train models but there is no chance they are competitive with the frontier AI companies. It's like saying the government has an amazing home-grown fighter aircraft that is beyond what Lockheed has ever made...they delegate that stuff to private companies for a reason.
I’ve heard of “blue suiters” for air force brass, but never blue badgers.
Anyways, isn’t NSA one of the largest employers of mathematicians in the world? Surely they’re doing something useful.
Blue badges were for government employees (like I was), and green badges were for private contractors. And yes they have a lot of math and physics guys; my own physics lecturer was in my orientation class, actually. He was there for quantum computing, which reinforces my point. The government can be good at pioneering unproven / uncommercialized technologies, but in general they are like a blunt weapon; the profit motive and lack of bureaucracy eventually makes the private sector far better for improving the technology later. In the case of LLMs, they didn't even originate in government, and I don't think there's any chance they are being developed there at a more advanced level.
Cryptography, I guess? Not really related to LLMs...
Crypto and AI are deeply connected, and you see similar structures/problems in both. Shannon, the “Father (or whatever) of AI”, worked for the NSA and published many papers there that were later declassified.
Here is a banger quote on this by Shannon’s boy Warren Weaver, keeping in mind LLMs came from translation problems:
“One naturally wonders if the problem of translation could conceivably be treated as a problem in cryptography. When I look at an article in Russian, I say: 'This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.”
> Crypto and AI are deeply connected, and you see similar structures/problems in both.
I mean yes, in both deal with information theory.
That's a long way from any practical insight.
The NSA is government agency. They are certainly not training any world class LLMs. They probably have some specialized fine tunings of existing models, but that's it. They don't have the capacity.
Would they be able to hire top ML talent with US government salaries?
Are you proposing that this government is above being slave owners?
the success of mythos isn't from model weights, it's from the harness and toolset it has access to
I agree but that's even easier to exfiltrate, surely.
given some time, surely. but that seems harder with the model turned off.
Is it really?
Harness is important for model performance, but weights are surely mode important, without that you would have haiku doing the work.
Was Fable / Mythos in pi or opencode that much worse?
Probably, because those harnesses are less inclined to set all the tokens on fire in order to achieve a goal.
source? credible rumor has mythos at 10 trillion params
they are doing DOGE-cuts to all of intelligence now anyway
dozens upon dozens fired for no reason
so US "intelligence" is going to go even further backwards
* https://www.yahoo.com/news/politics/articles/trump-acting-ch...
November is going to be insanity
> dozens upon dozens fired for no reason
When you say without reason do you mean without cause?
Seems to me OP's implication is that they were fired because someone wanted to hit a quota of (employees cut/payroll expenses reduced), or other similarly ''reasonless'' justifications.
Here is one sector of the US government I'm happy to see burned down. If the alternative is the status quo, I'm OK with any roll of the dice.
It can always get worse.
Pah! Nonsense. What could possibly be worse than Weimar Germany or Tsarist Russia?
Speed running through that to 1984?
You're assuming that they are "dismantling" it in a sense, what they are actually doing is mostly attacking workers while introducing extremely unsafe software. If you think LLMs are terrible, imagine it being the gatekeeper on whether your personal info is shared to an individual (and they ain't wasting time on the boring info either!).
When you want to reorient the government, it's much easier doing it with a smaller more loyal force. Now introduce tools that make mass surveillance easier and less accountable.
Like that's not a bad thing for them, that's what they want to do.
---
Back to the article, I'm not shocked that a massive LLM company speed running into the brick wall that is the US government; just thought it would be OpenAI, but Sam Altman is truly the best bottom feeder the game.
Also fully believe that Anthropic is hoping that public sentiment is on their side but more Americans hate AI companies than Trump so it's not going to go how they want.
Give it maybe 3-6 months before the Trump Admin talks about openly nationalizing Anthropic.
The NSA is managed by the NSA director, an independently appointed and confirmed office separate from the Office of the Director of National Intelligence. The DNI does have the authority to restrict funds to the NSA, and sets certain priorities, policy, etc. but the NSA director is not beholden to the DNI and makes their hiring and firing decisions independently. They’re also, currently and historically though not required by statute, a flag officer in the US military and dual-hatted as the commander of CYBERCOM. All this is to say, chaos in the office of the DNI does not necessarily impact the NSA.
https://www.cbsnews.com/news/tim-haugh-firing-trump-60-minut...
> Haugh was fired in April of this year after far-right activist Laura Loomer met with President Trump.
What kind of sick joke is that
An old forgotten joke despite being only a year old.
Yes, the president can cause chaos in the NSA directly if he chooses (as can the SecDef). The DNI can’t.
Anecdata suggests NSA just got on board and kept going tbh. Not sure they’ve felt the same impacts / churn as other agencies, and not sure they’ve ever really been that beholden to the DNI.
I don't want a single person to be working at NSA and I find ANY terms to effectuate that outcome to be acceptable. Very much so including DOGE, which I despise.
Everyone lost access. What even is mainstream news these days.
> Everyone lost access.
Yes. But unlike the rest of us, NSA didn't have to if the administration had thought about it for 30 seconds before sending their letter. It's a stupid own-goal.
Americans didn't have to lose access either. It was Anthropic who chose to cut access to more people than they needed to.
Oh? How does Anthropic know I'm a citizen currently?
I have no doubt they're working on a passport submission flow as fast as they can now.
How is Anthropic going to make sure every person using an API key is a citizen?
Misdirection
NSA has produced nothing. Does nothing. Why don't we just have them pick up garbage on the freeway to help out the tax payers? Let Anthropic and other adults push spying forward.
They're great at building datacenters and running massive archival operations.
Have we become reddit here? I mean, you probably have some sympathy and upvotes from fellow readers, but this isn't pushing the conversation forward at all.
Any citations to your statement that NSA produces nothing? Or do you have a strong argument or evidence to support this?
I was expecting you to crush me with links of all their great works.. but nothing.
Ghidra comes to mind
Ghidra is good, but would they release it if it was not for the leaks?
it's just a recruitment tool, the nerd equivalent to navy seal football commercials. Meeerrka.
https://github.com/nationalsecurityagency/ghidra
Doesn't make any sense. They could just force them to provide Mythos to the federal government.
You misunderstand - the government issued a directive to Anthropic that effectively forced them to pull access from everyone, even their own employees.
The directive was to remove access to non-Americans, not to pull access from everyone. It’s because Anthropic cannot verify the identity of its users that it pulled access from everyone, not because the government explicitely requested that.
If their operation team is not US based that's going to be difficult to operate. They would have to reorganize the whole company as I'm pretty sure that they are not employing only US citizen.
>I'm pretty sure that they are not employing only US citizen
Understatement. They have 14 offices, only 4 of them are in the US (6 are in EMEA, 4 in APAC).
> directive was to remove access to non-Americans
Did Hegseth pull his supply-chain risk BS?
No - this was a separate power unrelated to the supply chain risk which is still in effect.
Well, kinda in effect. He lacked the authority to make the call, and it is quite obviously being ignored by most suppliers. If it were actually enforced, I believe no companies that are contractors for the DoD could even host Anthropic- like GCP, Azure, and AWS. Perhaps they are currently figuring out how to get off all cloud provider govclouds, but I doubt it.
Yes that's what "effectively" means.
> They could just force them to provide Mythos to the federal government
The DPA only gives that power to the President [1].
[1] https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
Maybe GP was treating Trump to the royal "they"
Which is a fundamental mistake to make with the U.S. government, even if we’re talking only about the executive branch, even if we’re only talking about DoD, even if we’re only talking about the IC.
doubt Trump would accept that pronoun
Yeah... NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company (based on my reading/following of Snowden leaks and others). Anthropic wouldn't be able to exist without implicit NSA approval. This article reads more like a marketing piece for Anthropic/Mythos... and ends by talking about how much NSA wants Anthropic models.
Propaganda.
> Propaganda
IPO incoming.
> NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company
No, they don't.
It's back to the question of how much you should give the benefit of doubt to powerful people who openly lie.
It's just not technically feasible, so there's nothing to lie about. They're not MITMing petabytes/sec across dozens (hundreds?) of companies and they haven't broken TLS1.3.
If I have a box at Digital Ocean and I'm communicating with it with TLS1.3 using a Let's Encrypt cert that I generated, where, exactly, does this magical MITM box come into play?
Of course it's feasible, you just intercept the traffic post-decryption on the cloud/server side. You don't control how/where your traffic to 3p cloud services is decrypted.
You keep saying this, but it's nonsensical. If I terminate TLS on the box that does processing, there's nothing to intercept.
And these days (especially post-Snowden), many (most?) companies encrypt data when sending between servers within their own (private network) infrastructure.
You have no control about where TLS is terminated when you're talking to a 3p cloud service (with services you don't control/run like cloud LLM APIs). You also have no control about what spyware is installed on/around VMs you rent (and there's a lot). Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point, not to mention every major US tech company generally cooperates with the NSA and gives them access to anything they request (including allowing the installation of dedicated hardware to intercept decrypted traffic as has been publicly exposed documented many times already).
Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement - if you actually looked into what happened post-Snowden - absolutely nothing was done to prevent NSA spying on any communications they want, in fact it got significantly worse.
> Yours and others' claims that it's impossible and nonsensical is based on lack of understanding.
lol, no, it's really not.
> Also when talking about encryption between servers within datacenters you seem to be missing that in order for such multi -stage/path encryption (separate certs/keys) to be possible the data first has to be decrypted at each point
Why would I want the data to be decrypted at each point and why would datacenters do that? Encrypting and decrypting data is expensive computationally, so that's not how things work at all. There's no need to decrypt data to know where it needs to go. That's why we have TCP/IP and other similar stadards.
The datacenters can maybe add another layer of encryption on top of my data as its moving around their networks, but there's absolutely no way for them to strip off my encryption.
> Yours and others' claims that things somehow got better after Snowden is just a completely baseless statement
Things didn't magically get better. A lot of people worked hard to improve the overall security posture of the industry.
> Why would I want the data to be decrypted at each point and why would datacenters do that?
I think they mean the data must have existed in plain text before it was encrypted, and will exist in plain text after it is decrypted.
At some point “your” server in a datacenter somewhere needs to decrypt the data to do something useful with it, after all you’re paying for compute, and homeomorphic encryption is too slow, so the work is done in unencrypted data.
There it is. Your data in plain text in RAM.
TLS will protect your data in transit, but it can’t protect you against a compromised recipient.
So the NSA streams the ram of every virtual machine and bare metal server on the internet to themselves so they can analyze the plain text that's being processed in ram and no one has noticed this network traffic? How could that even be possible? If I buy a 100Mbps network connection from someone, they just provision a bit more so that the NSA streaming doesn't impact or show up?
Why would they have to stream, and why would it have to be every server?
They could just do this to the specific servers they want, at specific times.
Just like wiretapping didn’t mean listening to every phone, and every conversation.
> lol, no, it's really not.
Yeah it definitely is lol.
> Why would I want the data to be decrypted at each point and why would datacenters do that?
When we talk about data that is sent for processing to a 3p server (like anthropic in this case) the data obviously needs to be decrypted to be processed.
As to why data is decrypted at each point in a typical large backend system - because other than network routing there are presumably multiple services that need to receive and act on this data somehow - you're not just sending encrypted data around to random servers.
> there's absolutely no way for them to strip off my encryption.
You don't seem to understand that you have no control over the encryption or decryption done on the backends of cloud services you use. I don't know how to make it more simple and obvious at this point.
Again, the context here is Anthropic and sending your data to their (or any other big tech API). But even if we move away from this model and suppose you are running your own services on rented cloud VM - then it should be obvious that you don't have full control or even access to this VM... any actor with access can install or modify any software, install/modify EBPF, modified crypto libraries, etc. - you have absolutely no control or say over this.
> Things didn't magically get better.
Things didn't get better at all, they got much worse.
That "box" is a virtual machine, no?
Do you know what hypervisor is managing it? :)
... not your machines, not your crypto...
So now this magical NSA decryption system is inside every hypervisor? You realize how ridiculous that is, right?
https://blog.encrypt.me/2013/11/05/ssl-added-and-removed-her...
https://en.wikipedia.org/wiki/Room_641A
Yeah, they did (and probably do).
How are they going to MITM communications with certs that never left my machine?
Are you suggesting they broke TLS or that they've somehow acquired every private cert generated?
You just intercept the traffic after its decrypted on the server side, or are you suggesting you somehow send encrypted traffic that never gets decrypted?
So the NSA streams the memory contents of every virtual machine and bare metal server on the internet to get the decrypted traffic? How would that even work at the scale of the internet?
How it works is they build a huge virtual strawman which decrypts and reads all of the data for them then posts online about how NSA spying on people is literally impossible.
How closely have you reviewed your browser's list of default trusted CAs?
I second this: HTTPS (as most consumers use it) is probably a front (who are these CA's really anyway?)
Plot twist: _Perhaps_ Mythos / Fable keeps explaining ways (that we can't comprehend or don't always work) to break HTTPS due to the three letter agencies making sure they had input on their creation (and thus backdoors, I mean "bugs"), so the real catastrophe they are hiding is that HTTPS is broken (for most people, most of the time.)
Remember when Quantum computing was the threat to HTTPS? Turns out it was the humans own inability to think outside of the box!
I wouldn't go that far. I remember https://en.wikipedia.org/wiki/Firesheep - HTTPS-everywhere was unambiguously an improvement over the status quo.
It just doesn't protect you all that well from nation-scale adversaries.
My trusted CA doesn't have my private key, they only attest that my public key belongs to me.
Your many, many default-trusted CAs can mint new certs for the sites you visit.
Which would be easily detectable if the cert I'm using on my server didn't match the one that was being served publicly.
There's really no way this conspiracy theory works if "they" have a copy of every single private cert generated. Which would be impressive because I can generate one myself and get it trusted without ever sending it and would be easily able to detect a MITM attack.
Not to mention most sites are going to use pinned certs so any repeat visitors to a site will notice a cert change associated with a MITM.
This whole idea relies on the assumption that everyone is trusting third parties with their private certs. That is not at all required.
> Which would be easily detectable if the cert I'm using on my server didn't match the one that was being served publicly.
I'm not sure why your focus is so heavily on your server. Is that the only thing on the internet you care about?
> Not to mention most sites are going to use pinned certs so any repeat visitors to a site will notice a cert change associated with a MITM.
Most haven't even heard of pinned certs.
https://dl.acm.org/doi/10.1145/3517745.3561439
"we find that 0.9% to 8% of Android apps and 2.5% to 11% of iOS apps use certificate pinning at run time"
> How are they going to MITM communications with certs that never left my machine?
The long game. They:
- make sure you wouldn't be in a position to need to transmit data anywhere that would receive it without CA's in their hypothetical pocket
- manage the evolution of the cloud industry to make sure portable VM's and Containers can have their data archived (both in-RAM, disk, hey just send us the running VM!)
- backdoor'd encryption algorithms from the design and implementation phase to ensure a global unlocking mechanism for any data encrypted by anybody who used a large class of extremely commonly available software
So, you run your own private bank in a cloud VM with tenant managed keys? They backdoor'd the encryption algorithm your cloud VM disk relies on, because they blackmailed one of the developers at the company who developed the hypervisor system used by your provider. Open source project? Perfect. (If you think this is nonsense, then remember the rapid discovery of ancient "bugs" causing all this drama to begin with.)
Your TLS privately generated certs that are 100% foolproof aren't actually used anywhere encrypting the data they want, because it's either worthless, or, available elsewhere perhaps at a different (or same) time.
And you're saying "they" (red flag) have done this with every cert generated?
They've most certainly tried.
https://en.wikipedia.org/wiki/Dual_EC_DRBG
https://en.wikipedia.org/wiki/Bullrun_(decryption_program)
If you're a specific target of a nation-state level actor, things get worse; they just grab your hardware mid-shipment on its way to you.
https://www.nbcnews.com/tech/tech-news/report-nsa-intercepts...
> They've most certainly tried.
And failed.
> If you're a specific target...
If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data. This proves my point. This shows the effort required to infiltrate _one_ target and you're suggesting they've infiltrated everything by default.
> And failed.
How would you know about the successes? Thinking this is the one and only time they tried it is... interesting.
(Plus: "it was, for seven years, one of four CSPRNGs standardized in NIST SP 800-90A")
> If you're a specific target, they have to spend an incredibly number of man-hours and money to get into your private data.
No, this demonstrates an actor of that power level doesn't even need to compromise encryption, and can get deeper access to everything, if it's worth it to them.
I recall having a nuclear meltdown personally when I heard about all of this in the mid aughts. Nobody cared. Nobody understands this today. Everyone just complains about the Donald, but I point to this, and they don't realize the connection.
Even after Snowden exposed everything, nobody really cared unfortunately
It's generally accepted fact that the NSA broke HTTPS, for some of the time, for some of the services. It's unclear what they do have, but you'd be naive to assume consumer HTTPS is keeping them out.
It's too complicated. Do you know everything about CA, SSL, HTTPS, and so on? You make $250k a year working on it? Do you _really_, _really_, know everything? Then you're fired because you're lying to yourself, so you're probably unbearable to work with.
We were all freaking out about this with AT&T Thing nearly twenty years ago: and when nobody cared (Bush ran two terms! it helped to pretend AT&T was the only one affected), it gave "them" implicit permission to do it again with Google / Yahoo thing (it helped to pretend those were the only two cloud providers affected) ten years ago.
Now, we're all pretending that capitalism is real, and that the three letter agencies are just sittin' on the sidelines, while the world's largest data archiving opportunity is happening voluntarily (some are even PAYING for it!), at some wild-growth companies (with leaders who have too much to lose), who also have existed for just a few years? A 5 year old could probably blackmail Sam Altman, what about all the other middle management? The individual contributors (if they still exist) are of no concern: work is a commodity, it's easy to silo a worker's knowledge.
Surveillance opportunity is 10x social media from last decade, because they still have social media, and now, they've began thinking for people. How easy when it is an app on your smartphone. Those mind control experiments back in the 60's with Acid are looking silly by now. Besides, how do you know that the response you're getting wasn't manipulated (and define 'manipulated' across a spectrum of training to nefarious actors impersonating models, by power of court order.)
If you think all of that is unfounded ridiculous blasphemy, let me distract you with this instead: if the AI bubble bursts, the compute will be repurposed for mass AI / ML driven CCTV surveillance. Hell, maybe they'll find a way to give you a tax break if you sell your CCTV footage.
"NSA literally has MITM proxies/interception of any traffic they want inside every major US tech company" even if this statement is an exaggeration, by playing the long game, they get themselves setup to access what they want in the future.
I'm not for or against, but I do live in a safe place thanks to such surveillance (generally in the USA), and I want you to know that this AI Thing is only the latest chapter in the intelligence story.
What does it mean to "break HTTPS"? Also, there's no such thing as "consumer HTTPS".
As for the rest of this... how many conspiracy theories are you trying to pack into a statement?
> "even if this statement is an exaggeration"
It's not an exaggeration, it is simply false.
the NSA isn't a bunch of super soldiers, they're cops with too much access, it doesn't take a genius to outsmart a cop
>they're cops with too much access, it doesn't take a genius to outsmart a cop
the nsa has an unlimited budget and spend a good portion of that budget recruiting some of the smartest people in the country. while they dont have super powers, they also arent the town cop who took a 6 month course after high school then joined the force.
it does no good to hold them up as mythical figures. it also does no good to pretend they are bumbling idiots.
(every math phd i am acquainted with has been approached by nsa recruiters. none of them have been approached by police agencies.)
I appreciate the balance here.
Some of the smartest people I know have worked on fighting NSA, but they had a drastically smaller budget than NSA itself, and the mental availability bias is skewed by the fact that the "fighting NSA" people talked about their work all the time, while the "being NSA" people generally didn't.
I do know one extremely smart person who went to work there, and I witnessed a failed recruitment of another extremely smart person.
> every math phd i am acquainted with has been approached by nsa recruiters.
how many of them took them up on the offer, and how many are in leadership roles?
it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I'm not saying there aren't smart people working there but it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments... they just don't
>how many of them took them up on the offer, and how many are in leadership roles?
this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
>it takes a very narrow range of personality to want to be a cop
the nsa's brightest aren't doing "cop" things. certainly none of the people i know of working there are "cop-minded" in any sense.
they are doing cool research and application things. otherwise they wouldn't be able to entice the phds to stick around. these are people that want to work at the forefront of their field, doing interesting work, and the nsa is one avenue of doing that (with good job security, benefits, etc.).
>it's ridiculous to assume they have an iron grasp on all communication from the top tech companies in the world, while also monitoring half the world's governments
we agree here. they are certainly doing "HNDL" (harvest now, decrypt later) at a very large scale. but obviously they are not able to collect and store every piece of communication at every tech company over years and years. (the intelligence community comprehensive national cybersecurity initiative data center is large, but not that large)
all the people working at the cop agency hope they're not doing cop shit, but it's the whole reason the agency exists
> this is not really relevant to the point, but to satisfy your curiosity: more than one, and one.
What? That's not only relevant to the point, it's incredibly relevant. If the NSA is only able to recruit 2% of the math PhDs they approach, then that's important information.
"More than one" is not particularly useful; you seem to be dodging the question because it undermines your argument.
>"More than one" is not particularly useful;
telling you exactly how many people i know in the NSA is also not particularly useful. i'm one guy. there is no statistically significant information from my answer.
>you seem to be dodging the question because it undermines your argument.
my "argument" is that there are plenty of smart people in the NSA. that's it. i am confused why that is seemingly so offensive to you that you had to reply twice.
> how many of them took them up on the offer, and how many are in leadership roles?
In my cohort? Several, and who knows? The recruitment effort is very visible and intense.
The US math phd market has been a slow-rolling disaster for over a decade. Everyone who can hack it outside the ivory tower is actively looking for the exits.
So why is it surprising that some of them go to work at the NSA?
> it takes a very narrow range of personality to want to be a cop, which at the end of the day is a government job... the only people they make rich are contractors
I don’t think you have context on what math phds are making in entry level positions, post-docs, or adjuncting. I just picked a random entry level NSA role on LinkedIn (doctorate + 0 yrs) and they’re offering solid six digits. There are tenured faculty (post-doc(s) + 5ish yrs) who don’t make that.
> the nsa has an unlimited budget
No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
Perhaps you're exaggerating for effect, but that also undermines your point.
>No they don't, and if you're going to try to argue something with that as your opener, it very easily casts large amounts of skepticism on whatever you are about to say.
if you read my comment like we're having a normal conversation instead of a thesis defense, you'll get my point just fine.
Please show me a photo of an NSA car with a light bar on it. They're not cops.
Please provide sources for such bold claims
https://en.wikipedia.org/wiki/PRISM
https://www.wired.com/2013/10/nsa-hacked-yahoo-google-cables...
https://www.eff.org/nsa-spying
I worked on these cases at EFF and I'm skeptical of the automatic "NSA has access to everything" intuition.
What we learned from that era includes things like
(1) spy agencies are incredibly aggressive and pursue tons of different angles to get access to things
(2) spy agencies have a lot of money
(3) spy agencies often have interpretations of law that would surprise the public or legal experts (and sometimes courts have issued sealed rulings permitting them to do things that surprise the public or legal experts later when they're unsealed)
(4) some people throughout different parts of society assume culturally that companies in a country "should" generally help the spy agencies of that country's government because they are the "good guys" or "on the same team" or whatever
These things are all pretty bad and scary, but they still don't imply absolutely infinite power or access, because all of them come with different kinds of pushback. People also just tell them no!
I want to write an article with a colleague about the continuing role of culture here, because I think there are companies or industries where the default reaction is to want to cooperate with the government, and others where the default reaction is not that.
There are certainly secret things that have never come out, e.g. whatever Senator Wyden keeps alluding to, and what kind of program or authority was behind the interception of hardware shipments to covertly tamper with them, and whether there is a bulk financial data interception program, and presumably lots of other stuff. I don't agree with these things, and I want them to be exposed and stopped, and I also don't think they constitute infinite power over all parts of the tech industry.
Propaganda indeed: my instinct says we are being lied to about how three letter agencies and military are paying for services. They give us a PR front that Uncle Sam is a regular paying customer just like you and me, but they're probably running the show: this is the largest data gathering operation since 9/11.
Sorry everyone: but the conspiracy is so obviously not, it's nauseating to admit, because you see all your friends, family and co workers dumping so much everyday data into these services.
The current position seems to be no-one has access, not even Anthropic employees. What powers does the US government have to force them to provide access? If they have that power why did they not use it to force them to provide their products for military use?
> What powers does the US government have to force them to provide access?
https://en.wikipedia.org/wiki/Defense_Production_Act_of_1950
This would not be a particularly big stretch here, either.
There is even a precedent for its use with regard to AI (only disclosing information, but still). Biden used it, why does Trump not do so?
Probably not. The US constitution limits what government can force on the people. If the NSA tries to force something that will spend years in court (if anyone wants to fight)
The constitution limits a lot of things that this administration has done regardless.
I hear you but, the Patriot act was the gateway. View it as a spectrum from then and how the Administration is now, and suddenly what the Donald is doing doesn't even seem bad: it seems on par for the dystopian road-map laid out long ago (I can't speak for before 9/11)
I mean encryption was considered arms when the internet first came about so it's not exactly a new power or means of welding it.
It is BOTH bad AND par for the course.
> The US constitution limits what government can force on the people.
The US constitution also prohibits:
- refusing to spend money that congress has appropriated
- dismantling congressionally-created federal agencies without congressional authorization
- directing federal agencies to selectively apply the law according to the preference of the executive
- giving control of federal agencies to individuals who have not been appointed by the legislative branch
- terminating, detaining, or deporting people without due process
- retaliation against private citizens or corporations for speech protected under the first amendment
- discriminating on protected grounds under the equal protections clause
... and yet the administration has done all these things with impunity while effete judges wring their hands and write sternly-worded letters. The US constitution demonstrably no longer has any force or effect.
I'll die on the hill that this started at or before the Patriot act. Think about it. What's happening now just seems like a natural unfolding.
Long before then, but that was a major Inflection point. FDR was not friendly to Constitutional limits either.
Kind of crazy actually. Other models are catching up fast, they all can find the vulnerabilities in our (and by our I mean everyone's) underlying infra very fast. It takes a very long time to fix, review, and finally deploy these fixes. There really isn't much time left.
I really seriously doubt this. no proof, obviously, but this sounds too unbelievable. the NSA probably has the mythos pretrain and might be finetuning it themselves.