Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic is expanding to Colossus2. Will use GB200

twitter.com

Post reply on HN
Print view View original

211–220 of 367 posts

Re: Anthropic is expanding to Colossus2. Will use GB200

#211

Post by watwut »

hackinthebochs wrote: post #157
giancarlostoro wrote: post #130

Earlier quoted context omitted.

One thing I do like about Grok is that it makes it stupid easy to see what its referencing, and gives you the links to those resources. Which most models sometimes either don't bother, or don't do much of a good job of doing. It's not the top model, but it is definitely high up there, people's blind rage for anything Elon Musk is the only reason most people don't realize how capable it is unfortunately. Grok is not e…

For conversational or general knowledge questions I also much prefer Grok. Musk's vanity aside, it is much less censored than the other frontier models.

Where less censored means "censors facts and left leaning claims while actively promoting far-right lies".

Re: Anthropic is expanding to Colossus2. Will use GB200

#212

Post by Glohrischi »

urams wrote: post #20

Very possible Elon is doing this to make give Anthropic better chances against OAI while he attempts to reshape xAI. Also possible he sees infra as the future of xAI if he really believes in the value of space compute. Hard to see this any of this as anything other than a bearish sign for Grok though.

That would be absolutly ludicris.

His own product is competing against anthropic.

Re: Anthropic is expanding to Colossus2. Will use GB200

#213

Post by Glohrischi »

fillskills wrote: post #47
urams wrote: post #20

Very possible Elon is doing this to make give Anthropic better chances against OAI while he attempts to reshape xAI. Also possible he sees infra as the future of xAI if he really believes in the value of space compute. Hard to see this any of this as anything other than a bearish sign for Grok though.

We could see the first company vertically integrated from etching to chip to data center

We wont for anytime soon.

Intel and TSMC are not what they are today just because they buy a very expensive EUV Machine from ASML but because they have the knowledge and infrastructure to even use these machines.

Re: Anthropic is expanding to Colossus2. Will use GB200

#214

Post by Glohrischi »

virgildotcodes wrote: post #52

Here's a probably stupid question - if someone were unbounded by ethics and conceivably had enough power and connections to power to shield themselves from many consequences of their actions - and that person owned these DCs, could they in theory observe all the streams of tokens coming in and out of these models, and even exfiltrate copies of these models wholesale to have their own teams do what they will with them…

Yes he could do that and I thought about this too. Very weird tbh

Re: Anthropic is expanding to Colossus2. Will use GB200

#215

Post by rdtsc »

Kelteseth wrote: post #155
gaze wrote: post #18

I think that's the datacenter with the gas turbine generators that operate without permits because they're "portable." Data centers have tremendous externalities but colossus is a particularly nasty offender, and not just due its size. Edit: They did it with Colossus and now they're doing the exact same thing with Colossus2. https://www.selc.org/news/xai-built-an-illegal-power-plant-t...

So for a guy that literally has a company that produces batteries and solar panels, choose to use gas turbines. So much for saving the planet.

> So for a guy that literally has a company that produces batteries and solar panels, choose to use gas turbines

“Sorry, we have to shut down your database because it’s been overcast for the last two days”

Now, they could have a mix, of course, but just running on solar and batteries at that power is not realistic.

Re: Anthropic is expanding to Colossus2. Will use GB200

#216

Post by stingraycharles »

tristanj wrote: post #62
virgildotcodes wrote: post #52

Here's a probably stupid question - if someone were unbounded by ethics and conceivably had enough power and connections to power to shield themselves from many consequences of their actions - and that person owned these DCs, could they in theory observe all the streams of tokens coming in and out of these models, and even exfiltrate copies of these models wholesale to have their own teams do what they will with them…

1) The situation you described would be covered under the contract between Anthropic and xAI, and that any violation of that would be subject to financial penalties and legal proceedings. The US has a robust corporate legal system, and disputes do get resolved through the court system, although in a slow and costly manner. The contract can stipulate a penalty at a high enough amount to discourage this behavior. 2) Ou…

> There is no reason do this, because there are far better ways to exfiltrate data from Anthropic models. Chinese companies are already doing this at an industrial scale where they are reselling Claude tokens for 10-20% of the cost while retaining the data to train their own models.

I think you missed the part where Anthropic stopped displaying their thinking tokens over the past few months, and instead now provides “summarized thinking”, letting Haiku summarize Opus’ thoughts.

So it is now much more difficult (impossible?) to distill the models.

I also think you over-estimate how well the legal systems works in the US nowadays, and under-estimate just how much power Elon has in the government.

Re: Anthropic is expanding to Colossus2. Will use GB200

#217

Post by Maken »

renticulous wrote: post #196
tristanj wrote: post #191

Earlier quoted context omitted.

They tried and failed. xAi made a mistake building Colossus 1 and ended up with heterogenous cluster of H100/H200/GB200 GPUs. This is a nightmare to train huge models on because each card has different specs, features, and hardware requirements. During gradient synchronization, a heterogeneous cluster would bottleneck on the slowest GPU (H100) so the faster GPUs would end up idling. They also probably ran into unexpe…

How can something so obvious be overlooked by team building the data centre? Can't the sharding be uneven so that weaker GPUs still finish fast by taking on a smaller workload?

It's not like they had much of an option, when everybody was hoarding every GPU they could. For the second Colossus they could book future production, but the first one had to be built ASAP so xAI looked as a serious competitor in the AI space.

Re: Anthropic is expanding to Colossus2. Will use GB200

#218

Post by aurareturn »

ACCount37 wrote: post #200
aurareturn wrote: post #188

Earlier quoted context omitted.

Pretty sure models are encrypted all the way.

Can't run inference on encrypted weights and get any kind of performance out of it.

The whole system has encryption all the way through.

Otherwise, OpenAI/Anthropic would never use external clouds since the weights are some of the most valuable assets in the world.

Re: Anthropic is expanding to Colossus2. Will use GB200

#219

Post by akimbostrawman »

KptMarchewa wrote: post #183
akimbostrawman wrote: post #139

Earlier quoted context omitted.

Opposed to all other models being the bastion of objectivity? Must be truly vindicating to have to hear other peoles opinions after decades in the silicon valley bubble.

This comment is very similar to what russian propaganda does. It's not aimed at convincing you to support them, but to convince you everyone is lying and there is no meaningful difference between each position, so you stay apathetic.

Yeah yeah every opinion you don't share or like is a russian bot or literally hitler.

Re: Anthropic is expanding to Colossus2. Will use GB200

#220

Post by jbs789 »

shenberg wrote: post #165
tristanj wrote: post #89

Earlier quoted context omitted.

That's very doubtful given recent news that Colossus is running at 11% capacity and has hundreds of thousands of idle GPUs. xAI acquired too many GPUs and currently doesn't have enough customers to use them. That's why they are making compute deals with Anthropic and Cursor. xAI is bleeding money and this compute deal with Anthropic will pay for all of xAI's capex ($25 billion) in 2 years.

11% MFU does not mean 89% of GPUs are idle, it means that they're using the GPUs ineffectively.

The post you’re referring to doesn’t even include “89%” or “idle”. I’m not sure I understand the point you’re making then rebutting.
Post reply on HN