Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic’s $5B, 4-year plan to take on OpenAI

techcrunch.com

Post reply on HN
Print view View original

51–60 of 504 posts

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#51

Post by raverbashing »

seydor wrote: post #29
raverbashing wrote: post #23

Earlier quoted context omitted.

The secret is not scraping PII in the first place (which is not really difficult, though it requires some planning)

not sure if it can ever be possible. i can ask chatGPT to do stylography analysis on our comments, find our other accounts and go from there. I'm pretty sure most pieces of human-generated data is identifiable at this point

This is not how it works, (unless of course you're pretending and collecting all these 'auxiliary data' on purpose) and even if it was, there's still plenty of non-PII data around

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#52

Post by nl »

alexb_ wrote: post #35

If someone released a chatGPT/characterAI with NSFW content enabled it would eat into a big share of their users (and for characterAI, maybe take all of them). Seriously, look into what people are posting about when it comes to characterAI, and it's 80% "here's how to get around NSFW filters". Unsure why nobody is taking this very very obvious hole in AI tech.

It's trivial to fine tune llamma to be NSFW if that's what you want.

But there's an entire universe of much more interesting apps that people don't want NSFW stuff in. That's why most foundation models filter it out.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#53

Post by jprete »

qikInNdOutReply wrote: post #36

If its THE exponential curve of AGI, every day not invested, will result in years behind in a few months. So these are rather small investments, but still bigger then the "to little to late" of the european union. Its not very visual or intuitive, but in some games, were the resource curve is exponential, small early headstarts become whole armies, were the opponent fields none in very short time. Especially as AGI i…

In the real world there aren’t any actual exponential curves, they’re all sigmoids where the observer doesn’t see the slowdown yet.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#54

Post by dfjklseldklk »

qikInNdOutReply wrote: post #36

If its THE exponential curve of AGI, every day not invested, will result in years behind in a few months. So these are rather small investments, but still bigger then the "to little to late" of the european union. Its not very visual or intuitive, but in some games, were the resource curve is exponential, small early headstarts become whole armies, were the opponent fields none in very short time. Especially as AGI i…

This assumes that you can't steal information to catch up quickly, or that progress made isn't easy to copy once it's obvious that it works.

A big part of why chatgpt is a big deal is that it shows that the overall approach is worth pursuing. Throwing stupid numbers of GPUs at a problem you don't know will be solvable is hard to justify. It's easy to throw money at a problem you know is solvable.

Nuclear weapons are the prime example of this: Russia caught up both by stealing information and just by knowing fission was possible/feasible as an explosive.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#55

Post by lhl »

omneity wrote: post #16
seydor wrote: post #8

Earlier quoted context omitted.

Sure, let's make an EU commercial LLM. Let's start by scraping all the Francophone internet. Then let's remove all the PII data and potentially PII-data. Easy-peasy. Then let's train our network so as not to spew out or make up PII data - easy peasy Then let's make it able to delete PII data that it has inadvertedly collected on request. Simultaneously it should be recording all the conversations for safety reasons.…

BLOOM has been trained on a 3M€ grant from French research agencies CNRS and GENCI. Doesn’t have any of the constraints you’re talking about.

BLOOM's training corpus ROOTS did make some efforts at removing PII https://arxiv.org/pdf/2303.03915.pdf btw, but AFAICT that was not at the behest of the French government.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#57

Post by blueboo »

didgeoridoo wrote: post #12
Havoc wrote: post #6

>“tens of thousands of GPUs.” I find the focus on GPUs a little odd. I would have thought that at 5 billion / 4 year scale ASIC route would be the way to go GPUs presumably come with a lot of unneeded stuff to play crysis etc

Do GPUs give you more flexibility to take different approaches? Maybe they’re paying extra for optionality. Or maybe (most likely) TechCrunch is using the term “GPU” imprecisely.

To programmer_dude’s point, compute center GPUs don’t have hardware for the rasterisation stage, which is a particularly inflexible bit of the graphics pipeline. Omitting it and emphasising the geometry (matrix multiplication) capabilties is meant to give it more flexibility/less of a strongly-opinionated graphics focus.

As for the “GPU” term, it’s a bit of a historical relic, presently it serves as a useful indicator of compute hardware (in contrast to CPU and Google’s TPU.) Nvidia itself calls its A100 a “Tensor Core GPU.”

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#58

Post by potamic »

dpflan wrote: post #44

"Dario Amodei, the former VP of research at OpenAI, launched Anthropic in 2021 as a public benefit corporation, taking with him a number of OpenAI employees, including OpenAI’s former policy lead Jack Clark. Amodei split from OpenAI after a disagreement over the company’s direction, namely the startup’s increasingly commercial focus." So Anthropic is the Google-supported equivalent of OpenAI? Isn't the founder going…

What does a policy lead do and how are they relevant to an early stage startup? I would be more interested in seeing which researchers and engineers join.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#59

Post by igravious »

Taywee wrote: post #41
programmer_dude wrote: post #15

Earlier quoted context omitted.

There are AI / datacenter focused GPUs (like the A100, H100 etc.). They do not have any graphics rendering circuitry.

Does it seems silly to anybody else to even call these GPUs? That's a GPU minus the G.

Agreed. I guess it's because of the architectural heritage but at this point GPU is something of a misnomer.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#60

Post by rvz »

rsp1984 wrote: post #37

AI models are in a race to the bottom and everybody inside Anthropic knows it. Besides OpenAI, with billions to spend plus a partnership with MS, there's also Google, Apple, Meta and Amazon who can afford to run losses on AI for years without blinking an eye. And if that wasn't enough the Open Source world is releasing new models almost weekly now, for free. Anthropic is putting on a big show to convince gullible inv…

Correct. Stability.ai (Stable Diffusion), Apple (Won't be surprising to see them announce on-device LLMs with Apple Silicon support), Meta (LLaMa), etc are already at the bottom and at the finish line with their AI models given for free.

O̶p̶e̶n̶AI.com will eventually have to raise their prices which is bad news for businesses not making enough money and still are sitting on their APIs as O̶p̶e̶n̶AI.com themselves are running up huge costs for their AI models in the cloud for inferencing.

Anthropic is just waiting to be acquired by big tech and the consolidation games will start again.

Post reply on HN