Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic’s $5B, 4-year plan to take on OpenAI

techcrunch.com

Post reply on HN
Print view View original

451–460 of 504 posts

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#451

Post by selectodude »

bugglebeetle wrote: post #449
rnk wrote: post #438

Earlier quoted context omitted.

I've been looking into buy a mac for llm experimentation - 64, 96 or 128gb of ram? I'm trying to decide if 64gb is enough, or should I go to 96gb or even 128gb. But it's really expensive - even for an overpaid software engineer. Then there's the 1 or 2 tb storage question. Apple list price is another $400 for that second tb of storage. For 64gb of ram, you can get an m2 pro, or get 96gb which requires the upgraded cp…

I’m pretty sure you can get a purpose-built pc tower in that range. Why would you favor a Mac over that? A lot of this stuff only has limited support for MacOS.

The unified GPU/CPU memory structure on ARM Macs is very, very helpful for running these LLMs locally.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#452

Post by nerpderp82 »

mike_hearn wrote: post #313
anentropic wrote: post #288

If Apple would wake up to what's happening with llama.cpp etc then I don't see such a market in paying for remote access to big models via API, though it's currently the only game in town. Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent…

If I were Apple I'd be thinking about the following issues with that strategy: 1. That RAM isn't empty, it's being used by apps and the OS. Fill up 64GB of RAM with an LLM and there's nothing left for anything else. 2. 64GB probably isn't enough for competitive LLMs anyway. 3. Inferencing is extremely energy intensive, but the MacBook / Apple Silicon brand is partly about long battery life. 4. Weights are expensive t…

Running models locally is the future for most inferencing cycles. There is a lot of accuracy that could be improved in your numbered list trying to dissuade people.

> 64GB probably isn't enough for competitive LLMs anyway

I am trying to charitable, but this is pretty not true. And the hedging in your statement only telegraphs your experience.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#453

Post by lsy »

lhl wrote: post #284
carlmr wrote: post #168

Earlier quoted context omitted.

My question would also be what kind of insight McKinsey can provide here. What, if anything, do they know about AI that we don't know?

You don't need to just take one source. OpenAI authored their own paper [1] on the economic impacts of just LLMs: "Our findings reveal that around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted." Goldman Sachs Research just pushlished their own analysis as well. [2] Their conclu…

From [1]: "In our study, we employ annotators who are familiar with LLM capabilities. However, this group is not occupationally diverse, potentially leading to biased judgments regarding LLMs’ reliability and effectiveness in performing tasks within unfamiliar occupations."

From [2]: "Analyzing databases detailing the task content of over 900 occupations, our economists estimate that roughly two-thirds of U.S. occupations are exposed to some degree of automation by AI."

These are people who do not understand the jobs they are claiming AI will do. Ultimately, I think they are not doing much better than guessing.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#454

Post by boringuser2 »

anentropic wrote: post #288

If Apple would wake up to what's happening with llama.cpp etc then I don't see such a market in paying for remote access to big models via API, though it's currently the only game in town. Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent…

You need state of the art consumer tech to run a model comparable to GPT-3 locally at a glacial pace.

Or, you can use a superior GPT 3.5 for free.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#455

Post by scotty79 »

thundergolfer wrote: post #255
Cthulhu_ wrote: post #223

I just want to know where those billions go to. Cloud server running costs?

The all-in cost of their technical talent probably exceeds $500k/year pp, excluding stock compensation.

If they narrow down their training data to software while keeping the size of the model they might shed that cost really quickly.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#456

Post by cubefox »

nemothekid wrote: post #329
cubefox wrote: post #130

Earlier quoted context omitted.

It is amazing that in the year 2023, where things are possible that were science fiction until recently, we still rely on private payment processors, credit card companies, which extract fees for a service that doesn't have any technical necessity anymore. I think the reason is just inertia. They work well enough in most cases, and the fees aren't so high as to be painful, so there is little pressure to switch to som…

> we still rely on private payment processors, credit card companies, which extract fees for a service that doesn't have any technical necessity anymore The technical necessity is there; for your chase-backed visa card to pull money from chase and deposit it into your shop's citibank, there needs to be some infrastructure. Whether a private company or the government provides this infrastructure is another story. (Alt…

Bank transfers are a thing. They don't require an intermediary credit card company. The problem is that currently such a transfer is usually slow because of software/protocol issues.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#457

Post by ukuina »

visarga wrote: post #213
wokwokwok wrote: post #165

Earlier quoted context omitted.

It’s astonishing to me that people seem to believe the llama models are “just as good” as the large models these companies are building, and most people are only using the 7B model, because that’s all their hardware can support. …I mean, “not-bad-at-all” depends on your context. For doing mean real work (ie. not porn or spam) these tiny models suck. Yup, even the refined ones with the “good training data”. They’re to…

You know what I believe is also a toy model? chatGPT Turbo, you can tell by the speed of generation. And it works quite well, so small size is not an impediment. I expect there will be an open model on the level of chatGPT by the end of the year because suddenly there are lots of interested parties and investors. Eventually there will be a good enough model for most personal uses, our personal AI OS. When that happen…

I think you're conflating speed of inference/generation with optimization. gpt-3.5-turbo does not fit on a single GPU unlike the "toy" models.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#458

Post by ukuina »

aunty_helen wrote: post #132
jerrre wrote: post #115

Earlier quoted context omitted.

I don't think the grain market is growing as fast as the AI market

Don’t confuse the ai market with the foundational llm model market. Think of LLMs as the understanding component in the brain, once you can understand instructions and what actions need to happen from those instruction you’re done. The rest is integrations, the arms legs and eyes of langchain. Then memory and knowledge from semantic search, vector databases and input token limits.

This is the real answer.

The LLM is but the core of the entire ecosystem. Just like how MLOps is 99% of the work, choosing an LLM is 1% of the effort in the final product.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#459

Post by evilduck »

bugglebeetle wrote: post #449
rnk wrote: post #438

Earlier quoted context omitted.

I've been looking into buy a mac for llm experimentation - 64, 96 or 128gb of ram? I'm trying to decide if 64gb is enough, or should I go to 96gb or even 128gb. But it's really expensive - even for an overpaid software engineer. Then there's the 1 or 2 tb storage question. Apple list price is another $400 for that second tb of storage. For 64gb of ram, you can get an m2 pro, or get 96gb which requires the upgraded cp…

I’m pretty sure you can get a purpose-built pc tower in that range. Why would you favor a Mac over that? A lot of this stuff only has limited support for MacOS.

You can't get that much VRAM on a PC for a comparible price.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#460

Post by two_in_one »

dpflan wrote: post #44

"Dario Amodei, the former VP of research at OpenAI, launched Anthropic in 2021 as a public benefit corporation, taking with him a number of OpenAI employees, including OpenAI’s former policy lead Jack Clark. Amodei split from OpenAI after a disagreement over the company’s direction, namely the startup’s increasingly commercial focus." So Anthropic is the Google-supported equivalent of OpenAI? Isn't the founder going…

"public benefit".. Ah, they are so good. May be they will _open_ something after all.. ;)
Post reply on HN