Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi-K3 on HuggingFace

huggingface.co

Post reply on HN
Print view View original

501–510 of 588 posts

Re: Kimi-K3 on HuggingFace

#501

Post by veber-alex »

fnordpiglet wrote: post #445
chrsw wrote: post #405

Earlier quoted context omitted.

Then there’s the questions of token efficiency and token quality.

I have to say cc opus 5 is abysmal. It talks to itself incessantly, gets stuck in minutia, fails to understand problems clearly and makes steering mistakes constantly. It also has a weird behavior where it says “ok I know exactly what to do and I will start now,” then sits waiting for user input. If you’re not on the ball you’re constantly losing 5m/1h cache. Just give me back 4.6.

I still see Opus 4.6 available in CC.

Also I have been using Opus 5 for the last few days and I find it work fine. It's a little verbose but the code quality is good.

Re: Kimi-K3 on HuggingFace

#502

Post by Imustaskforhelp »

hnfong wrote: post #491
cosmojg wrote: post #478

Earlier quoted context omitted.

IANAL, but probably not, at least not in the United States. Under U.S. copyright law, the weights of machine learning models are excluded from copyright as they are the product of an automated optimization process (e.g., stochastic gradient descent, expectation maximization, genetic algorithms) rather than human authorship. Granted, this has yet to be fully tested in court and going to court is expensive, so it's lik…

If your argument is true, what would make LLM weights not copyrightable, but compiled code copyrightable?

(IANAL), but the argument would be the same because how compiled code (binary data) is copyrightable but the code (binary data) of an image of a painting created by say a monkey itself with no human involvement isn't.

As such as they have mentioned in the argument, their argument is sound in terms of the level of human involvement in creation of the artifact.

Re: Kimi-K3 on HuggingFace

#503

Post by minimaltom »

lightningdev1 wrote: post #458
minimaltom wrote: post #452

HF says activation function is "SiTU-GLU", but I can't find any info on that? Anyone know / is this a typo? Even the python code for inference seems to use normal activations.

It stands for Sigmoid Tanh Unit Gated Linear Unit. Check the tech report/paper (page 7): https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_repo...

Thanks! I had a clanker graph it with sliders, its a very interesting function to play with! seems like it has the negative 'hump' typical of a GELU coupled with the saturation of a sigmoid.

Its interesting that saturation is ideal at this scale, i thought we were all in on self-activating functions (i.e. swish) but in fairness all those papers lacked ablations at scale nor discussion of stability.

Re: Kimi-K3 on HuggingFace

#504

Post by adgjlsfhk1 »

barbacoa wrote: post #285
walrus01 wrote: post #50

Earlier quoted context omitted.

It will be very interesting to see what kind of 'slow' performance people get from running it on a no GPU, but tons of RAM server (like a dual or quad socket xeon with 1.5 to 3TB of RAM). For the purpose of giving it longer duration tasks to generate a piece of something and come back and check on what it has done in 4 or 6 hours. Even if the output is like 5-6 tok/s, that might be usable for some purposes. Huge pric…

They are saying that AMD's new Epyc Venice CPU has 16 memory channels allowing up to 1.6Tb/s of bandwidth. Which is higher bandwidth than most non-HBM GPUs. So full CPU local AI inference may become viable option in coming years.

the GPU competition is using 16 gpus, so the actual comparison is that the CPU has <1/10th the bandwidth

Re: Kimi-K3 on HuggingFace

#505

Post by flutas »

mdasen wrote: post #309
walrus01 wrote: post #64

Earlier quoted context omitted.

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. And not everyone's electricity is equally expensive, I pay about $0.075 USD per kWh. It would for example cost me about $48 a month of electricity (not counting cost of cooling) to run a quad socke…

That's an unusually low electric rate for the US - way below the lowest state average which is Idaho at 12.4 cents. It's certainly possible that you are getting 7.5 cents including delivery, but I've had friends say that they're "getting 13 cents per kWh" here in Massachusetts, but that's just the supply rate and the delivery is another ~18 cents. There are parts of states like Grant County Washington that have cheap…

I'm in Arkansas and get rates fairly similar as quoted.

From my last bill

> KWH USAGE 2590 - $183.37

There's a base customer cost of $18 on top of that, but yeah ~$0.077/kWh taxes included.

Re: Kimi-K3 on HuggingFace

#506

Post by Imustaskforhelp »

embedding-shape wrote: post #485
mock-possum wrote: post #392

Earlier quoted context omitted.

Is that even enforceable?

Before figuring that out, could Facebook take you to court in order to argue their case that it is enforceable, and thereby forcing you to get lawyers and be distracted by the preparation and all that comes with this?

Nothing stops anybody from suing anybody else (and maybe even winning) though. What Napster was doing in isolate was just a technology yet RIAA and others sued and the lawsuit had led to the conclusion that the tech could be held reliable and if what users were doing were an intentional known to the tech-creators.

So the mere knowing of it led them to lose it and Napster died because of that but also the actual nail in the coffin was that they couldn't significantly do anything to the problem about that given its P2P nature, Ipods were around the same time and RIAA was a bit afraid of that too but Steve jobs assured them that because of the walled garden they could better control the piracy issue and have proper ways of countering it.

Now aside from the interesting details of that time I showed, coming to my main point, Lawsuits can sometimes happen for lesser reasons than or just limited to plain and simple license violations and if a company is earning 20 Million dollars supposing so, then they might also have a really good lawyer insurance package and could lawyer up just as well.

The core argument lies on proving if AI weights are copyrightable or not from my understanding because the licenses could be best applied under copyright material not public domain materials and the other discussion[0] by @cosmojg shows the most likely cases of AI not being copyrightable?, so you would have to prove if AI is copyrightable or not.

Now that would be a fun lawsuit to watch though.

[0]: https://news.ycombinator.com/item?id=49074087

Re: Kimi-K3 on HuggingFace

#507

Post by Aeroi »

"Native Multimodality & Long Context: Kimi K3 understands text, images, and video within the same model, and supports a 1-million-token context window."

Video will be interesting. It's the most context rich medium combined with sound, movement ect. We need more evals and benchmarks around video understanding. It's the key to unlocking truly great physical intelligence. I've been working a bit on this @camerasearch and its challenging.

Re: Kimi-K3 on HuggingFace

#508

Post by mambru »

flutas wrote: post #505
mdasen wrote: post #309

Earlier quoted context omitted.

That's an unusually low electric rate for the US - way below the lowest state average which is Idaho at 12.4 cents. It's certainly possible that you are getting 7.5 cents including delivery, but I've had friends say that they're "getting 13 cents per kWh" here in Massachusetts, but that's just the supply rate and the delivery is another ~18 cents. There are parts of states like Grant County Washington that have cheap…

I'm in Arkansas and get rates fairly similar as quoted. From my last bill > KWH USAGE 2590 - $183.37 There's a base customer cost of $18 on top of that, but yeah ~$0.077/kWh taxes included.

Is that for a month or a year?

Re: Kimi-K3 on HuggingFace

#509

Post by rootusrootus »

mindslight wrote: post #469
rootusrootus wrote: post #431

Earlier quoted context omitted.

My point is that the tradeoff to get off-peak pricing is that on-peak is way, way more expensive. So you can charge the EV off-peak to maximize the savings, but everything else you do during on-peak time costs way more. Using myself as an example: I adjust my A/C to run outside of 5pm-9pm (peak) if at all possible, we try to avoid pointless high-draw usage during that same window, and both of our EVs hold off chargin…

If you leave the EV charging out of your consumption, does a time of day plan still save you money on the remaining usage? Or does it cost you? If it saves you money, then it would make sense to be on a ToD plan regardless of EV charging. Which means it makes sense to consider your additional EV draw as costing the marginal off-peak rate. Essentially the EV load has the valuable property of being dispatchable. You ca…

> If you leave the EV charging out of your consumption, does a time of day plan still save you money on the remaining usage? Or does it cost you?

It would cost me more (modestly so, less than 10%) to be on TOD without the EVs. This will vary by customer, of course, and I expect that the power company designs TOD to be a wash for the average customer. They even guarantee it won't be more than 10% more expensive over the first year or they will refund the difference.

Post reply on HN