Live data from Hacker News

Understanding LLM Compute Requirements

1–3 of 3 posts

Post by pgspaintbrush » Sun, May 28, 2023, 10:47 PM UTC

Re: Understanding LLM Compute Requirements

Post by pgspaintbrush » Sun, May 28, 2023, 10:47 PM UTC

A guide for the non-technical

Post by brucethemoose2 » Mon, May 29, 2023, 12:37 AM UTC

Inference is not that flop intense for modern IGPs... the bigger issue is the sheer memory requirements.

Training requires a ridiculous amount of both, but again the raw compute requirements would be much lower with huge memory pools.