Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic’s $5B, 4-year plan to take on OpenAI

techcrunch.com

Post reply on HN
Print view View original

341–350 of 504 posts

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#341

Post by sebzim4500 »

mike_hearn wrote: post #313
anentropic wrote: post #288

If Apple would wake up to what's happening with llama.cpp etc then I don't see such a market in paying for remote access to big models via API, though it's currently the only game in town. Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent…

If I were Apple I'd be thinking about the following issues with that strategy: 1. That RAM isn't empty, it's being used by apps and the OS. Fill up 64GB of RAM with an LLM and there's nothing left for anything else. 2. 64GB probably isn't enough for competitive LLMs anyway. 3. Inferencing is extremely energy intensive, but the MacBook / Apple Silicon brand is partly about long battery life. 4. Weights are expensive t…

>One possibility is simply cost: if your device does it, you pay for the hardware, if a cloud does it, you have to pay for that hardware again via subscription.

Yeah but in the cloud that cost is ammortized among everyone else using the service. If you as a consumer buy a gpu in order to run LLMs for personal use, then the vast majority of the time it will just be sitting there depreciating.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#342

Post by sebzim4500 »

horsawlarway wrote: post #340
mike_hearn wrote: post #313

Earlier quoted context omitted.

If I were Apple I'd be thinking about the following issues with that strategy: 1. That RAM isn't empty, it's being used by apps and the OS. Fill up 64GB of RAM with an LLM and there's nothing left for anything else. 2. 64GB probably isn't enough for competitive LLMs anyway. 3. Inferencing is extremely energy intensive, but the MacBook / Apple Silicon brand is partly about long battery life. 4. Weights are expensive t…

> but it's hard to see situations where the local approach beats out the cloud approach. I think the most glaring situation where this is true is simply one of trust and privacy. Cloud solutions involve trusting 3rd parties with data. Sometimes that fine, sometimes it's really not. Personally - LLMs start to feel more like they're sitting in the confidant/peer space in many ways. I behave differently when I know I'm…

Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. I find it hard to believe that there are enough people willing to pay a large premium to run these locally to make it worth the R&D cost.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#343

Post by samstave »

anentropic wrote: post #288

If Apple would wake up to what's happening with llama.cpp etc then I don't see such a market in paying for remote access to big models via API, though it's currently the only game in town. Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent…

Can we re-invent SETI with such LLMs/new GPU folding/whatever hardware and re-pipe the seti data through a Big Ass Neural whatever you want to call it and see if we have any new datapoints to look into?

What about other older 'questions' we can point an AI lens at?

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#344

Post by MisterPea »

nickpp wrote: post #123
seydor wrote: post #8

Earlier quoted context omitted.

Sure, let's make an EU commercial LLM. Let's start by scraping all the Francophone internet. Then let's remove all the PII data and potentially PII-data. Easy-peasy. Then let's train our network so as not to spew out or make up PII data - easy peasy Then let's make it able to delete PII data that it has inadvertedly collected on request. Simultaneously it should be recording all the conversations for safety reasons.…

The moment Europe decided to regulate tech, it decided in effect to stagnate. Innovation and creativity are incompatible with regulation. Unfortunately for us, tech is where progress happens currently. Europe is being left behind. Not that it was very competitive in the first place anyway.

While true, I think they do innovate in policy around it. Regulation is an ever-evolving field as well and they do think about it more.

But yes, in a half-century I'm very curious where Europe will be. India passed the UK in gdp recently and Germany sooner or later.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#345

Post by [deleted] »

anentropic wrote: post #288

If Apple would wake up to what's happening with llama.cpp etc then I don't see such a market in paying for remote access to big models via API, though it's currently the only game in town. Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent…

[deleted]

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#346

Post by mike_hearn »

horsawlarway wrote: post #340
mike_hearn wrote: post #313

Earlier quoted context omitted.

If I were Apple I'd be thinking about the following issues with that strategy: 1. That RAM isn't empty, it's being used by apps and the OS. Fill up 64GB of RAM with an LLM and there's nothing left for anything else. 2. 64GB probably isn't enough for competitive LLMs anyway. 3. Inferencing is extremely energy intensive, but the MacBook / Apple Silicon brand is partly about long battery life. 4. Weights are expensive t…

> but it's hard to see situations where the local approach beats out the cloud approach. I think the most glaring situation where this is true is simply one of trust and privacy. Cloud solutions involve trusting 3rd parties with data. Sometimes that fine, sometimes it's really not. Personally - LLMs start to feel more like they're sitting in the confidant/peer space in many ways. I behave differently when I know I'm…

People have trusted search engines with their most intimate questions for nearly 30 years and there has been what ... one? ... leak of query data during this time, and that was from AOL back when people didn't realize that you could sometimes de-anonymize anonymized datasets. It hasn't happened since.

LLMs will require more than privacy to move locally. Latency, flexibility and cost seem more likely drivers.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#347

Post by mike_hearn »

bootsmann wrote: post #339
mike_hearn wrote: post #313

Earlier quoted context omitted.

If I were Apple I'd be thinking about the following issues with that strategy: 1. That RAM isn't empty, it's being used by apps and the OS. Fill up 64GB of RAM with an LLM and there's nothing left for anything else. 2. 64GB probably isn't enough for competitive LLMs anyway. 3. Inferencing is extremely energy intensive, but the MacBook / Apple Silicon brand is partly about long battery life. 4. Weights are expensive t…

> Even if a high end MacBook can do local inferencing, the iPhone won't and it's the iPhone that matters Doesn't the iPhone use the local processor for stuff like the automatic image segmentation they currently do? (Hold on any person in a recent photo you have take and iOS will segment it)

Yes but I'm not making a general argument about all AI, just LLMs. The L stands for Large after all. Smartphones are small.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#348

Post by Analemma_ »

cubefox wrote: post #130
xena wrote: post #40

Earlier quoted context omitted.

The main reason why companies don't allow NSFW content is because of puritan payment processors that see that stuff and then go absolute sicko mode and lock people out of the traditional finance system.

It is amazing that in the year 2023, where things are possible that were science fiction until recently, we still rely on private payment processors, credit card companies, which extract fees for a service that doesn't have any technical necessity anymore. I think the reason is just inertia. They work well enough in most cases, and the fees aren't so high as to be painful, so there is little pressure to switch to som…

> I think the reason is just inertia.

It is not just inertia; it is government malice. The government loves that there are effectively only two payment processors, because this lets them exercise policy pressure without the inconvenience of a democratic mandate.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#349

Post by horsawlarway »

sebzim4500 wrote: post #342
horsawlarway wrote: post #340

Earlier quoted context omitted.

> but it's hard to see situations where the local approach beats out the cloud approach. I think the most glaring situation where this is true is simply one of trust and privacy. Cloud solutions involve trusting 3rd parties with data. Sometimes that fine, sometimes it's really not. Personally - LLMs start to feel more like they're sitting in the confidant/peer space in many ways. I behave differently when I know I'm…

Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers. I find it hard to believe that there are enough people willing to pay a large premium to run these locally to make it worth the R&D cost.

Having done a lot of Bank/Gov related work... I can tell you this

> Almost everyone is willing to trust 3rd parties with data, including enterprise and government customers.

Is absolutely not true. In it's most basic sense - sure... some data is trusted to some 3rd parties. Usually it's not the data that would be most useful for these models to work with.

We're already getting tons of "don't put our code into chatGPT/Copilot" warnings across tech companies - I can't imagine not getting fired if I throw private financial docs for my company in there, or ask it for summaries of our high level product strategy documents.

Re: Anthropic’s $5B, 4-year plan to take on OpenAI

#350

Post by mike_hearn »

anentropic wrote: post #336
mike_hearn wrote: post #333

Earlier quoted context omitted.

1. You're working backwards from a desire to buy more RAM to try and find uses for it. You don't actually need more RAM to use LLMs, ChatGPT requires no local memory, is instant and is available for free today. 2. Why would anybody be satisfied with a 64GB model when GPT-4 or 5 or 6 might even be using 1TB of RAM? 3. That may not be the case. With every day that passes, it becomes more and more clear that large LLMs…

1. You're working backwards from a desire to buy more RAM to try and find uses for it. I'm really not I had no desire at all until a couple of weeks ago. Even now not so much since it wouldn't be very useful to me But the current LLM business model where there are a small number of API providers, and anything built using this new tech is forced into a subscription model... I don't see it sustainable, and I think the…

I think llama.cpp will die soon because the only models you can run with it are derivatives of a model that Facebook never intended to be publicly released, which means all serious usage of it is in a legal limbo at best and just illegal at worst. Even if you get a model that's clean and donated to the world, the quality is still not going to be competitive with the hosted models.

And yes I've played with it. It was/is exciting. I can see use cases for it. However none are achievable because the models are (a) not good enough and (b) too legally risky to use.

Post reply on HN