Kimi-K3 on HuggingFace

Post by pizza234 » Mon, Jul 27, 2026, 9:26 PM UTC

walrus01 wrote: post #64

fooker wrote: post #63

Earlier quoted context omitted.

> Even if the output is like 5-6 tok/s, that might be usable for some purposes. You'll spend ~100x more on electricity than the API cost to have it run on someone else's GPU at several hundred tokens per second. I think some sort of extreme data privacy requirement is the only situation that justifies this, but the intersection of {needs absolute data privacy, needs to run SOTA model, cannot afford GPUs} is really re…

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…

Post by hemant6488 » Mon, Jul 27, 2026, 9:31 PM UTC

someguyornotidk wrote: post #54

pmg1991 wrote: post #30

Hoping no issues on Huggingface due to download rush.

For huge models like these, the only reasonable way to host them is via torrents. I don't understand why hf doesn't offer this as an option. Linux distributions got this right: Offer both HTTP and Torrents. Let the user decide.

Post by mindslight » Mon, Jul 27, 2026, 9:37 PM UTC

rootusrootus wrote: post #509

mindslight wrote: post #469

Earlier quoted context omitted.

If you leave the EV charging out of your consumption, does a time of day plan still save you money on the remaining usage? Or does it cost you? If it saves you money, then it would make sense to be on a ToD plan regardless of EV charging. Which means it makes sense to consider your additional EV draw as costing the marginal off-peak rate. Essentially the EV load has the valuable property of being dispatchable. You ca…

> If you leave the EV charging out of your consumption, does a time of day plan still save you money on the remaining usage? Or does it cost you? It would cost me more (modestly so, less than 10%) to be on TOD without the EVs. This will vary by customer, of course, and I expect that the power company designs TOD to be a wash for the average customer. They even guarantee it won't be more than 10% more expensive over t…

Post by serial_dev » Mon, Jul 27, 2026, 9:48 PM UTC

fnordpiglet wrote: post #445

chrsw wrote: post #405

Earlier quoted context omitted.

Then there’s the questions of token efficiency and token quality.

I have to say cc opus 5 is abysmal. It talks to itself incessantly, gets stuck in minutia, fails to understand problems clearly and makes steering mistakes constantly. It also has a weird behavior where it says “ok I know exactly what to do and I will start now,” then sits waiting for user input. If you’re not on the ball you’re constantly losing 5m/1h cache. Just give me back 4.6.

Post by bravoetch » Mon, Jul 27, 2026, 9:57 PM UTC

mdasen wrote: post #309

walrus01 wrote: post #64

Earlier quoted context omitted.

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…

That's an unusually low electric rate for the US - way below the lowest state average which is Idaho at 12.4 cents. It's certainly possible that you are getting 7.5 cents including delivery, but I've had friends say that they're "getting 13 cents per kWh" here in Massachusetts, but that's just the supply rate and the delivery is another ~18 cents. There are parts of states like Grant County Washington that have cheap…