Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K3 exploited the latest Redis server

twitter.com

Post reply on HN
Print view View original

41–50 of 106 posts

Re: Kimi K3 exploited the latest Redis server

#41

Post by baby_souffle »

NorwegianDude wrote: post #38
mohsen1 wrote: post #30

Earlier quoted context omitted.

Do you think any programmer really understands how their program works end-to-end? At some abstraction layer, we're all clueless. There are many layers between what you type into the text editor and the actual CPU ticks that make your program work. I bet nobody fully understands the whole stack. Now that that text editor accepts English, we're all calling each other names, etc.

While most don't, there are developers who actually do understand systems from end to end. Most developers are terrible developers, compared to the really good ones.

> While most don't, there are developers who actually do understand systems from end to end

The last time intel made a CPU where the entire thing could fit in one persons head was probably the 8086 or thereabounts.

There is no one developer or engineer at apple that understands everything going on from key press to bits moving around to print the characters I see when I type in this text box.

Re: Kimi K3 exploited the latest Redis server

#42

Post by [deleted] »

pizza234 wrote: post #26
theplumber wrote: post #25

Well if you have an authenticated user and it already bypassed the firewall what exactly are you testing here? These kind of “zero days” pretty pointless. Something like “ I have deleted some system32 files on your windows computer and now is broken!” Even I am already an admin user on the machine.

No, that’s a flawed analogy, because this is an RCE. It's (like) having credentials for a network service, then exploiting that service to execute arbitrary code on the Windows host (of course the blast radius depends on the underlying permissions). Access to Redis is not access to the underlying server.

[deleted]

Re: Kimi K3 exploited the latest Redis server

#43

Post by angry_octet »

illliillll wrote: post #14

This is a deeply uninteresting example for anyone clueful. It’s an authenticated RCE in redis, anyone even vaguely familiar with the codebase knows to not expect there to be any real security boundary in place here. Don’t confuse this with an unauthenticated RCE, that would actually matter. Absolutely anyone can shit out endless bugs like this with AFL, this is an extremely messy unhardened surface that expects trust…

Sorry but this is simply delusional. We're in an era where creds can be bought in online markets, and redis open source has no default requirement for password complexity.

If you can 'shit out endless bugs like this with AFL' then maybe redis isn't fit for purpose. But I think it more likely that it just needs hardening and an end to these laissez faire attitudes.

Re: Kimi K3 exploited the latest Redis server

#44

Post by teravor »

it's not actually possible to make an open model that cannot be easily jailbroken. when you have access to its entire state it's trivial to gaslight it into a non-refusal state (you can forge its responses to build up the jailbroken state).

I don't know if the sole kimi k3 provider gives that level of access atm however (where you can dictate its own responses to it).

Re: Kimi K3 exploited the latest Redis server

#45

Post by justinhj »

btown wrote: post #6

> this is the first llm that is capable and willing to write an exploit An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price). Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arb…

If the AI tools can easily find exploits, cannot those same tools be used to harden security? In fact the companies have an advantage over script kiddies: access to more expensive models and compute time as well as professional security engineers

Re: Kimi K3 exploited the latest Redis server

#46

Post by alightsoul »

btown wrote: post #6

> this is the first llm that is capable and willing to write an exploit An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price). Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arb…

This already happened with a closed galaxy model from openai attacking huggingface. Hugging face had no choice but to use the dangerous open source models you're talking about, glm 5.2. At this point you have to fight fire with fire.

Re: Kimi K3 exploited the latest Redis server

#47

Post by alightsoul »

Xalutiono wrote: post #8
walrus01 wrote: post #7

Earlier quoted context omitted.

This is a concern but given its size, it's also going to cost a potential user $500-600k in hardware to self host and run Kimi K3 at any useful speed with full context size. It's not something that just anyone interested in attacking a system can use. The size/cost of hardware is far beyond even something like a self-hosted GLM5.2 Q8 at approx. 850GB GGUF file on disk size, which can run at a slow tok/s rate on a ser…

Why would you caculate 500k? if Kimi is around 1-3tb big, even current DDR5 prices are at 15k.

Because you need GPUs to run it fast enough for an attack to be effective. 8 GPU servers with enough VRAM are that expensive.

Re: Kimi K3 exploited the latest Redis server

#48

Post by QuadmasterXLII »

justinhj wrote: post #45
btown wrote: post #6

> this is the first llm that is capable and willing to write an exploit An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price). Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arb…

If the AI tools can easily find exploits, cannot those same tools be used to harden security? In fact the companies have an advantage over script kiddies: access to more expensive models and compute time as well as professional security engineers

It appears not.

Re: Kimi K3 exploited the latest Redis server

#49

Post by CapsAdmin »

sureglymop wrote: post #18
btown wrote: post #6

> this is the first llm that is capable and willing to write an exploit An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price). Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arb…

What makes someone a script kiddie or not a script kiddie? Imo this differentiation is totally pointless now. It gives everyone with access to AI the ability to use it as intended but also otherwise. But that is true for the defensive side also.

> What makes someone a script kiddie or not a script kiddie?

The way I see it, a script kiddie is someone who buy weapons with the intention to cause harm. They don't know or care how the weapon works, as it's just a tool for their malicious plan.

A non-script kiddie would be someone who build weapons.

If you assume weapons have recreational purposes, the builder/non script-kiddie is less likely to be malicious as they love the craft of building weapons. For example recreating historical weapons, or just being nerdy about machinery of a gun works.

A builder with malicious intent as first priority can exist, they are more likely to just be interested in building.

So basically, nerds are perceived to be more ethical than non nerds.

Re: Kimi K3 exploited the latest Redis server

#50

Post by asveikau »

justinhj wrote: post #45
btown wrote: post #6

> this is the first llm that is capable and willing to write an exploit An open-source Kimi is going to have real economic impact (and not only because of its forcing function on frontier labs to indefinitely subsidize their models to meet a race-to-the-bottom market price). Because it's also putting sophisticated zero-day-seeking tools in the hands of script kiddies who can develop and run novel exploits against arb…

If the AI tools can easily find exploits, cannot those same tools be used to harden security? In fact the companies have an advantage over script kiddies: access to more expensive models and compute time as well as professional security engineers

Having the AI tools find exploits seems like a potential first step towards having the exploits be fixed.
Post reply on HN