Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the new Raspberry Pi AI Hat supercharges LLMs at the edge

blog.novusteck.com

Post reply on HN
Print view View original

11–12 of 12 posts

Re: How the new Raspberry Pi AI Hat supercharges LLMs at the edge

#11

Post by throwaway81523 »

What a crap article. The only actual info in it is that there are two models of the accelerator, and their prices, and that one is 2x faster. It gives the speed in TOPS but that is useless since it doesn't say what the operations are.

Others mention the article itself looks AI generated. I didn't spot that, but it would explain some things.

Re: How the new Raspberry Pi AI Hat supercharges LLMs at the edge

#12

Post by telgareith »

bhouston wrote: post #10
telgareith wrote: post #4

Earlier quoted context omitted.

Answer: smol, and a fraction.

The 1B to 3B models should be runnable on Raspberry PI 5 with 8GB. That seems reasonable. Probably equivalent to the new Apple Intelligence that runs locally on device in terms of performance if you can use the same model.

You're joking right? a rpi5 isn't in the same league as Apple SoCs.
Post reply on HN