[flagged]
Launch HN: General Instinct (YC P26) – Frontier models on edge devices
11–20 of 20 posts
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#12Sorry if this is somewhat off-topic:
Through my estimations, based on Bonsai's parameters/GB ratio, if one model were to have this ratio and Gemma4:12b's size, it would have the nice number of 54.125b parameters (that could run on 16GB of RAM). Is there any organization attempting something of this kind?
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#13I'm still kind of surprised that people are targeting edge deployment of MoE models. By definition they optimize for computation cost at the expense of memory efficiency. We generally need the opposite on the edge. I'm hoping to see more work in the other direction with cyclic/looped transformers and other memory dense approaches.
[flagged]
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#14As an aside, General Instinct sounds like the name I'd give a megacorp in one of my cyberpunk ttrpg campaigns.
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#15Sorry if this is somewhat off-topic: Through my estimations, based on Bonsai's parameters/GB ratio, if one model were to have this ratio and Gemma4:12b's size, it would have the nice number of 54.125b parameters (that could run on 16GB of RAM). Is there any organization attempting something of this kind?
Yes Google. They just released their Gemma 4 12b quant.
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#16You've likely heard about this - he'd probably like to talk to you and might potentially give you some good PR. https://www.youtube.com/watch?v=rAzT5lcezPs&t=467s
I assume PewDiePie runs something like DeepSeek 4 Flash on that rig.
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#17Hi Guanming/Bill. Would love to chat about what you're doing for actually running the models. I'm in a similar space, speeding up the `docker pull` component of inference deployment on edge devices (among other things!) If you're interested, shoot me an email at kyle@clipper.dev
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#18Sorry if this is somewhat off-topic: Through my estimations, based on Bonsai's parameters/GB ratio, if one model were to have this ratio and Gemma4:12b's size, it would have the nice number of 54.125b parameters (that could run on 16GB of RAM). Is there any organization attempting something of this kind?
Yes Google. They just released their Gemma 4 12b quant.
[dead]
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#19As an aside, General Instinct sounds like the name I'd give a megacorp in one of my cyberpunk ttrpg campaigns.
In cyberpunk ad world, after the campaign the whole town is littered with GI tracts.
Re: Launch HN: General Instinct (YC P26) – Frontier models on edge devices
#20How many watts? How does it effect power envelope?