Anthropic’s $5B, 4-year plan to take on OpenAI

Post by LargoLasskhyfv » Wed, Apr 12, 2023, 5:37 PM UTC

mike_hearn wrote: post #313

anentropic wrote: post #288

If Apple would wake up to what's happening with llama.cpp etc then I don't see such a market in paying for remote access to big models via API, though it's currently the only game in town. Currently a Macbook has a Neural Engine that is sitting idle 99% of the time and only suitable for running limited models (poorly documented, opaque rules about what ops can be accelerated, a black box compiler [1] and an apparent…

If I were Apple I'd be thinking about the following issues with that strategy: 1. That RAM isn't empty, it's being used by apps and the OS. Fill up 64GB of RAM with an LLM and there's nothing left for anything else. 2. 64GB probably isn't enough for competitive LLMs anyway. 3. Inferencing is extremely energy intensive, but the MacBook / Apple Silicon brand is partly about long battery life. 4. Weights are expensive t…

Post by LargoLasskhyfv » Wed, Apr 12, 2023, 6:44 PM UTC

worik wrote: post #401

nickpp wrote: post #185

Earlier quoted context omitted.

> Plenty of greedy people in non-capitalist systems. Totally agreed. But I am not placing any moral value on either greed or capitalism. I would think, however, that capitalists would not ignore such an obvious profit center as the sex industry. Thus my bafflement.

> But I am not placing any moral value on either greed or capitalism That is a missed opportunity * Capitalism: A system where who owns resources matters more tan who needs them is a morally bankrupt system. A system where starvation and homelessness is an acceptable outcome * Greed. Greed is bad for everybody. Concentrates scarce resources where they are not needed, that too is moral bankruptcy

Post by anentropic » Wed, Apr 12, 2023, 9:40 PM UTC

rnk wrote: post #490

anentropic wrote: post #481

Earlier quoted context omitted.

The unified memory ought to be great for running LLaMA on the GPU on these Macbooks (since it can't run on the Neural Engine currently) The point of llama.cpp is most people don't have a GPU with enough RAM, Apple unified memory ought to solve that Some people have it working apparently: https://github.com/remixer-dec/llama-mps

Thank you, that's exactly what I was looking for, specific info on perf.

Post by sharemywin » Wed, Apr 12, 2023, 10:28 PM UTC

Rufbdbskrufb473 wrote: post #471

sharemywin wrote: post #330

Earlier quoted context omitted.

would be nice if you could pay in your own token.

I'm not sure to what you're referring by "your own token", but most do offer a range of popular tokens by using one of the 3rd party payment providers like Coingate. I had paid for my servers with some Litecoin I have that I usually use for small purchases because of the low fees.

Post by leadingthenet » Thu, Apr 13, 2023, 3:07 PM UTC

nl wrote: post #462

kranke155 wrote: post #444

Earlier quoted context omitted.

Yeah I don’t see that happening anymore. Crypto is always going to have fees and off-ramps, although I do think it’s helped create competition in the transfer space. The real way crypto will work or not work is programmable money. If that works that will be huge, if it doesn’t then maybe someone will pick it back up 50 years from now.

The problem in this specific space is that many people don't want their payments to a NSFW company to be associated with them. Most blockchains makes this trivially traceable by design. The ones that don't (eg Tornado cash) end up being used for money laundering so on/off ramps won't touch them. We'll see what happens with the ZK-based chains, but this seems a systematic problem that is difficult to fix.