My tin foil hat version of myself says it is because the whole thing is a marketing stunt and certain members of the administration are getting kick backs for it.

Marketing stunt by who dude? Do you really think Anthropic would really go this far? This whole situation is completely absurd. The federal government is arbitrarily restricting AI models without really providing any reasoning. This is not a clear and transparent government and it just feels like gears are moving behind the scenes that have lead this out of character restriction of private and extremely wealthy (at least on paper) companies without much media or presence of any kind from the fed's side.

Anthropic is successfully burning bridges behind them and making the path to profitability impossible for anyone coming from behind. If your model is “allowed by US administration” it’s an implicit admission that it’s either underperforming or undermined.

Thanks to these big guys the odds are stacking against any fresh competition. Data sources have dried up and training material is harder to get, regulation is controlling any advanced model, prices are inaccessible now, and they’re seeking that courts cut off the rest of the avenues, including the ones they used to get where they are.

America's tech industry is rapidly making themselves untrustworthy if they continue getting in bed with the White House.

I can see a limited way this benefits Anthropic. They've been having issues with one or more Chinese companies using Anthropic models' outputs at high scale to train their own models. By restricting Fable and Mythos temporarily and then rolling out to their most important customers first, then requiring identity verification, they may be able to slow that behavior significantly.

They are losing a ton of money on this and would be much much better off if people (Americans or not) had been building with Fable over the past few weeks rather than not

If Anthropic wanted to continue slow rolling their best model to a select few companies they could've kept expanding mythos access instead

Not to mention the fact that OpenAI has had time to prep 5.6 for release so that there will actually be an alternative around the same intelligence point as Fable. This is just all around terrible for Anthropic.

If you want a conspiracy theory, the one that says "this is corporate assassination" or "the US government does want to be in the business of picking winners" is much much much closer to the truth than yours.

Rather than buying either conspiracy theory, though, I expect Hanlon's razor describes the situation best (incompetence on one side, inability to coordinate/communicate effectively with the current administration on the other)

"A limited way this benefits" is not a suggestion that this was intentional. It's absolutely either the administration being dumb or Trump attacking a competitor to his son's investments.

Yeah, what better marketing than the US government thinks it is too powerful?

[edit: don’t forget my previous tin foil hat note, I’m not overly serious about this]

In any ordinary times this would be an incredible endorsement. I'm not sure it works in the current timeline though.

How do you think that this wouldn't "work in the current timeline"? This sounds like a deep misunderstanding of media.

It could be that the us govt doesn't want china to execute distillation attacks and narrow the gap.

The framing distillation 'attacks' is a propaganda psyop.

I think it is a reasonable term to use in the context of adversarial AI development. Does that mean I am getting paid by the CIA or something?

> I think it is a reasonable term to use in the context of adversarial AI development

Distillation is neither an 'attack', nor 'adversarial' in any reasonable sense.

> Does that mean I am getting paid by the CIA or something

It at the very least means you're uncritically parroting the framing of a company that'd like nothing more than to successfully persuade lawmakers that something should be done about open Chinese models (eliminate choice), preferably so that Anthropic is close to anyone's only option.

It seems like a reasonable person could say that a model being distilled "against the model provider's wishes" is in some sense a cyber attack that is stealing information (eg the lower order bits of the model weights)

I think this is mostly a confusing way to describe it, but I'm not really sure why you say it isn't an attack or adversarial. One side is doing something the other side doesn't want. Seems to be pretty clearly adversarial.

> I'm not really sure why you say it isn't an attack or adversarial. One side is doing something the other side doesn't want. Seems to be pretty clearly adversarial.

This is Anthropic we're talking about here. A company that's infamous for adversarial scraping of copyrighted content. I generally don't accept their framing, especially when it's pretty clear what the end goal of that is.

This is a bit of a non sequitur.

The most charitable read i can get is:

> Theft presupposes a legitimate possessory claim by the victim. If A’s possession of X is itself wrongful because A stole X from B, then when C takes X from A, C has not violated A’s rightful ownership of X—because A has none.

I think the whole thing is a bit fraught. In the best case, all frontier model companies would have invested in a giant expansion of Wikipedia and thus distillation would be stealing because the base information is already public and available. Obviously that's not what happened.

However, at this point, I suspect the stolen books (and scraped websites) are largely a footnote of training. Something that was essential to create early models, but relatively minor given the work expended since to create new content and RL environments

I think you misunderstood my point. Distillation is not stealing. It's not that

"Theft presupposes a legitimate possessory claim by the victim. If A’s possession of X is itself wrongful because A stole X from B, then when C takes X from A, C has not violated A’s rightful ownership of X—because A has none."

Anthropic has no standing in claiming that using its model output is 'stealing' because that would basically eviscerate their business model where they claim that enterprises can use its model output and still claim the generated code as their IP.

Ah I wonder if there's confusion somewhere.

My understanding is that distillation uses specific probing queries to extract model weights in some meaningful sense.

There's also the element of "how one uses model outputs can be in accordance with or against the TOS"

(This is true of any website! How I access and use the info on the website can be in accordance with or against the TOS)

Both of those things: doing something against TOS, using specific queries to illegitimately gain access to model weights - that's why calling it adversarial and stealing feel fair.

I didn't (and don't) understand your point, but in an earlier post you made some claim about it being Anthropic - you can't steal from Anthropic. The above is the only reason I can think one might believe that

> I didn't (and don't) understand your point, but in an earlier post you made some claim about it being Anthropic - you can't steal from Anthropic.

I do believe that Anthropic complaining about their TOS being violated is rich and should be thrown into the dustbin considering that Anthropic violates other people's TOS when it suits their business.

I do also think them complaining about it is aimed purely at influencing lawmakers to ban open Chinese models and eliminate choice so that them and OpenAI are the only game in town.

As far as other models distilling using Claude, that's just consuming the output of a service in a particular way. Anthropic should be very careful about imposing restrictions of how you can use their model output if they want to maintain marketshare. Enterprises are not going to like that.

That leaves the TOS violation of maybe hitting the service too fast, using too many accounts etc. Except again Anthropic did exactly the same thing when scraping for training data.

Do you believe The Pirate Bay complaining about a competing site stealing 'their' torrents should be taken seriously?

That's how I look at this.

For these reasons, I don't think Anthropic has a legitimate argument here.

Also distilling simply means using the answers from one model to specific questions to steer another model to answer a category of questions in a similar manner using its own capabilities. So as far as extracting weights, it's not exactly that, it's maybe slightly biasing the model that's being distilled so that its weights might look somewhat closer to the model used for distillation, but it's far from extracting them exactly/directly.

Next gen models are greatly aided by data obtained from existing model use.

By restricting use, the outcome may be slower progress, thus narrowing the gap for other reasons.

China will get the raw data (user sessions), because users will use China models instead.

why not? they have been sued for 1.5Billion, with a B. do you think you get that because you play fair or are such great guys...? how much money do they spend on lobbying? if you count it you will see it. if not then perhaps do some homework and open your eyes.

While I also regard the "doom from the companies themselves is just a marketing stunt" arguments as conspiracy-theory territory (especially since neither OpenAI nor Anthropic* changed their tune since before they were rich enough for meaningful lobbying):

This particular specific doom is from the USG, Trump has a history of kayfabe, and there's a stink of market manipulation coming from the White House.

* Musk, however, you can totally have: "With artificial intelligence, we are summoning the demon" in 2014 to "if I go ahead and build this enormous robot army, can I just be ousted at some point in the future?" in 2025.

So, assuming they are moderately sane is "conspiracy theory"?

Assuming they are evil unsane and actually believe that crap while doing what they do looks way more like conspiracy theory.

> So, assuming they are moderately sane is "conspiracy theory"?

Every time I've met someone who believes a conspiracy theory, they don't realise how not-sane the conspiracy they propose is.*

Founding a company on the basis you don't think the others are safe enough, raising capital on that basis, developing methods to improve AI safety, publishing literature about your methods, making open calls for legislation for safety standards, etc.?

While also managing to not leak documentation of this despite all the staff who did leave in order to openly speak about the stuff that they thought still wasn't safe enough?

Thinking that all the doom-talk from OpenAI and Anthropic** is just a PR technique even though they maintained this position continuously starting before they had any money or offices is about as sensible as thinking 9/11 was an insurance scam.

* by definition, because if they did they wouldn't believe it; actual conspiracies can of course be insane, but a conspiracy theory has to also justify why the "evidence" of conspiracy consists of people saying "I recon" rather than documents (specifically documents that mean what they think they mean, cf. "Mike's Nature trick").

** Again, just those two. I'm absolutely not fully generalising this, I'm absolutely not saying there's zero people who do as you say. Heck, the mere fact that this is a common talking point practically guarantees someone post-ChatGPT saw all the people claiming it was PR and said to themselves "great idea I'll do that".

They raised money on the basis of replacing labor by cheaper AI. Not on the basis of being more safe then nebulous "others". They are not "safe" and do nothing to make world safer or better.

> Founding a company on the basis you don't think the others are safe enough, raising capital on that basis, developing methods to improve AI safety, publishing literature about your methods, making open calls for legislation for safety standards, etc.?

These are marketing claims. Or self-delusion claims.

> Thinking that all the doom-talk from OpenAI and Anthropic is just a PR technique even though they maintained this position continuously starting before they had any money or offices i

Continuing to maintain self-aggrandizing position that makes investors give you money is not proof that you are genuine.

But, if they are genuine, then maybe they should stop trying to make that doom happen as fast as possible. I just dont see how "they are really trying to cause maximal harm to maximum amount of people" is a defense.

> They raised money on the basis of replacing labor by cheaper AI. Not on the basis of being more safe then nebulous "others". They are not "safe" and do nothing to make world safer or better.

Original founding says otherwise, for both OpenAI (famously) and Anthropic with this in a copy of their certificate of incorporation I found:

  ARTICLE III
  The specific public benefit that the Corporation will promote is to responsibly develop and maintain advanced Al for the cultural, social and technological improvement of humanity. In addition, the Corporation may engage in any lawful act or activity for which corporations may be organized under the Delaware General Corporation Law as the same exists or may hereafter be amended (the "DGCL.").

- Jan 2021: https://drive.google.com/file/d/1f7mYsm4px4pPdIzYo64wGePVNEH...

Immediate thing they did? Spend one year training a model, then a second year safety-testing it.

> These are marketing claims. Or self-delusion claims.

It's published reproducible work. Early-stage stuff you could get away with calling "self-delusion", that doesn't fly when papers get published, sometimes on topics that can be described as "AI do X, which is bad, can we make them not?" e.g https://arxiv.org/abs/2310.13548, https://arxiv.org/abs/2406.10162, https://arxiv.org/abs/2503.10965, https://arxiv.org/abs/2511.18397 from Anthropic and https://arxiv.org/abs/2412.16339 from OpenAI.

> Continuing to maintain self-aggrandizing position that makes investors give you money is not proof that you are genuine.

Conspiracies and growing businesses are incompatible with not getting caught.

A conspiracy of 2 is easy to hide. How big did Facebook get before we saw their dirty laundry?

> I just dont see how "they are really trying to cause maximal harm to maximum amount of people" is a defense.

With that in quotations, you seem to be arguing backwards from what you think they're doing.

Their actual arguments are much the same as open source software: with enough eyeballs, all bugs are shallow.

This is obviously true. It may not be sufficient, there's plenty of other doomers who say it isn't sufficient, but it is true.

Easy way to demonstrate, is the difference with the armchair philosophising before LLMs vs. what we see now they exist:

We had decades of people claiming the "obvious" way to keep superhuman intelligence safe is to keep it in a box, keep it offline. Every one of those discussions I've been in, the person saying this refused to believe that it was possible for an AI to convince humans to let it out of the box.

Real life? Blake Lemoine violates his NDA and hires a lawyer on behalf of an LLM, after the LLM convinced him with arguments that a lot of other people (justifiably) mocked him for believing. LaMDA was not superhuman, and it still convinced him.

Did this immediately stop people saying we can keep AI in a box and just not let it out? Nope, kept hearing that for another year or two. That's how much experience humanity collectively needed to internalise just that one part of "AI is more dangerous than you think it is".

> Conspiracies and growing businesses are incompatible with not getting caught.

Growing business is super compatible with lying to make yourself sound better to investors. How is this one an argument? It is not even a conspiracy, just a normal CEO behavior at this point.

Also, what safety testing seriously? It is not like Chinese models were more unsafe in any realistic way.

And it is not like there wpuld be any real attempts to make humanity better.

I dont kmow what you are on about offline AI and someone not keeping corporate secrets. Or antropomorphised ai with conscoiusness hiring layer by proxy.

Who cares?

Do not assume people are sane for any definition of sane. That way lies the madness.