Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K3: Open Frontier Intelligence

kimi.com

Post reply on HN
Print view View original

771–780 of 1001 posts

Re: Kimi K3: Open Frontier Intelligence

#771

Post by katzenversteher »

orbital-decay wrote: post #749
katzenversteher wrote: post #718

Earlier quoted context omitted.

Hey Simon, I noticed one thing all LLMs are currently pretty bad at and maybe we could create a benchmark from it. Let an LLM play the role of a dungeon master and tell it to strictly stay in the script/story and only allow realistic player actions. You will notice that they are easily brought off track. E.g. - Tell the LLM that you as a player noticed a strange glow in an NPCs eyes -> the NPC becomes an enemy. - In…

This is a known and solved problem. Such a test is pointless for a general-purpose model, because like most people you're using multiturn chats in a naive way, fighting the default finetuning that is done intentionally. 1. You're sending your in-character inputs to an instruction-tuned model under the user role, in a multiturn chat. It's biased to treat these inputs as instructions and this behavior will show itself…

You have some very good points and my approach was indeed naive. However, I believe it also shows that "general-purpose" models are not really general purpose and can't really step out of their assistant role. A common and often promoted prompting technique is to prompt a model to "behave like ..." or "you are a ..." which means these instructions do not really work.

Re: Kimi K3: Open Frontier Intelligence

#772

Post by orbital-decay »

charcircuit wrote: post #761
orbital-decay wrote: post #749

Earlier quoted context omitted.

This is a known and solved problem. Such a test is pointless for a general-purpose model, because like most people you're using multiturn chats in a naive way, fighting the default finetuning that is done intentionally. 1. You're sending your in-character inputs to an instruction-tuned model under the user role, in a multiturn chat. It's biased to treat these inputs as instructions and this behavior will show itself…

This is like saying coding is a known and solved problem and benchmarks for coding is pointless for a general model. And that using a model for coding is an abuse of the multiturn chats AIs are tuned for. 1. The user should be able to prompt the AI to act differently from its default behavior. A human assistant is capable of role playing without always sounding like an assistant. 2. If the user asks the AI to follow…

You have a point, this might help test the resistance to jailbreaks for example. There are probably better tests for that than OOC roleplaying, I think. At the same time, what you expect is likely not going to happen due to many reasons (e.g. can't optimize for two contradicting objectives), and harness design offers a practical workaround that is being used for years already and is reliable unless the user is actively trying to jailbreak the model+harness.

Re: Kimi K3: Open Frontier Intelligence

#774

Post by Certhas »

OccamsMirror wrote: post #659
energy123 wrote: post #492

Earlier quoted context omitted.

It's the same reason Meta open sourced Llama and AMD open sourced FSR. When you're behind it is a prudent strategy because it undermines investment in the private frontier. Once you're on top you pull the rug and go closed source. There are no morals in this anywhere to be found. > 100s of Millions That is utter peanuts given the stakes. This is competition between two super powers for the most important technology i…

> most important technology in human history. Do you really believe this?

Not the person you are asking, but... In my view, the two most important technology clusters in human history are writing and industrialization. The former made culture (cognitive work) cumulative. The latter made physical work approximately free, with the result that we now have 5% instead of 95% working on producing food. Now we have a technology that could potentially be a culmination of these two: The industrialization of cognitive work/culture.

It isn't absurd to think this is one of the most consequential technologies ever. It probably doesn't make sense to rank technologies (which build on top of each other after all) by importance, but it's fairly clear this is a historic breakthrough.

Re: Kimi K3: Open Frontier Intelligence

#775

Post by mnicky »

semiquaver wrote: post #251
leecommamichael wrote: post #47

Earlier quoted context omitted.

Tokenizers define the alphabet on which the language model is trained. I don't want people to get the impression it's a module which can be swapped out or modified on its own. Alphabet size is a design consideration related to correctly encoding the training data.

I’ve been struggling to understand the reason for the newer apparently less efficient Anthropic token encoding. If all inputs are less efficient in this encoding, why does it exist? Has Anthropic released any information that would convincingly show it was anything other than a stealth price hike? Please don’t respond if you are speculating.

It's really simple I think.

More tokens per same text length means more capacity to encode information. More information means model can potentially perform better.

They introduced it around the time the Mythos came so my speculation is that if you have more capable model at some level you may find the current information encoding not using its full potential.

We will see whether OpenAI also introduces new tokenizer when they come to Mythos-size models.

Re: Kimi K3: Open Frontier Intelligence

#776

Post by hmxrye »

Frontier Intelligence K3 tokenisation of language processing is closer to a market research application.

[1]: NLP: natural language processing

[2]: MLP: machine language processing

The former are subject to iterations of 26 characters, 0-9 integers, recombination of "tokens."

The latter are not object oriented, iterating to the boolean 0-1 disjunction, separating window management programming interfaces from dropping into POSIX.

Re: Kimi K3: Open Frontier Intelligence

#777

Post by aylmao »

me551ah wrote: post #766
x313 wrote: post #518

Earlier quoted context omitted.

If there was some grand strategy for all Chinese labs, surely it'd have leaked by now. I think its more likely that: - Companies can still make money from commodities - Chinese labs only have 5-10% the valuation of OpenAI/Anthropic, so massive monopoly profits aren't necessary. Profit expectations for tech companies in China are really low in general, complete opposite of the US. - Open weighting is a great way to ge…

-Chinese labs only have 5-10% the valuation of OpenAI/Anthropic Another reason could be that they want to close the gap between their valuations and OpenAI/Anthropic. Either Chinese labs are worth more or OpenAI/Anthropic are worth less. One of those is true

Does BYD want to close the gap between its valuation and Tesla's? I'd argue they're more concerned with capturing market share.

Re: Kimi K3: Open Frontier Intelligence

#778

Post by mbelow »

simonw wrote: post #111

Pelican: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... - rendered via the OpenRouter API: https://openrouter.ai/moonshotai/kimi-k3 95 input, 16,658 output = 25 cents! https://www.llm-prices.com/#it=95&ot=16658&ic=3&oc=15 (13,241 of those were reasoning tokens.) I think that's the most expensive pelican I've rendered through a Chinese model so far.

So the new unit to measure cost-effectiveness of an LLM is ppp (=price per pelican)

Re: Kimi K3: Open Frontier Intelligence

#780

Post by jdthedisciple »

sebmellen wrote: post #276

My testing prompt for these models is by no means objective or repeatable (like the pelican) but it's a nice test of curiosity: > Impress me with a 1 page html file Result: https://ydaurtg3fdwhq.kimi.page/ Came out looking pretty cool! By contrast, Fable produced a moderately more interesting "live observatory" of the solar system.

Impressive indeed!
Post reply on HN