Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Smaller, faster, safer: running Kimi and GLM at scale

blog.cloudflare.com

Post reply on HN
Print view View original

21–30 of 74 posts

Re: Smaller, faster, safer: running Kimi and GLM at scale

#21

Post by hamdingers »

hankbond wrote: post #3
brokenodo wrote: post #2

I was interested in reading this until my slop detector went off at the paragraph starting with “It's worth being precise about where the benefit comes from, because it isn't raw speed.” I love AI, but I really hate reading it.

I have had to stop commenting this because it would end up on 50% of the posts here. I really wish we could flag prose as ai-generated on here and just filter it out.

Drives me nuts that comments are held to a higher standard than submissions.

HN is for conversation between humans[1] (about AI generated blogspam, apparently)

1. https://news.ycombinator.com/newsguidelines.html

Re: Smaller, faster, safer: running Kimi and GLM at scale

#22

Post by serf »

dgellow wrote: post #12
hankbond wrote: post #3

Earlier quoted context omitted.

I have had to stop commenting this because it would end up on 50% of the posts here. I really wish we could flag prose as ai-generated on here and just filter it out.

Don’t stop commenting about it, if there is something we (the readers) can do is ensure it is seen as uncool to post slop content

fine but please realize that also just pushes people away from caring about the comment section at all, too.

I can only read so many (either pro OR against) ".. IT'S AI! ! !" comments before skipping the thread. I can't be the only one.

Re: Smaller, faster, safer: running Kimi and GLM at scale

#23

Post by serf »

hankbond wrote: post #7
gr_norm wrote: post #5

Earlier quoted context omitted.

LinkedIn (of all places!) announced a button for flagging this recently: https://www.linkedin.com/posts/hsrinivasan1_ai-slop-is-a-top... How well it would work on this site, I'm not sure.

Next up, LinkedIn starts using this feedback to train a classifier. They then announce an officially approved "not slop" classification only for LinkedIn Gold member posts. The classified posts have a wider reach due to everyone filtering out AI slop. Non-members automatically get bucketed in with the slop bc they don't pay to have the verified classifier run on them.

seems like at that point it'd be a bigger benefit for LinkedIn to provide the one and only model trained to subvert the human capability to detect slop using that new negative corpus.

then we're all screwed I guess. lol

Re: Smaller, faster, safer: running Kimi and GLM at scale

#24

Post by mannanj »

hamdingers wrote: post #21
hankbond wrote: post #3

Earlier quoted context omitted.

I have had to stop commenting this because it would end up on 50% of the posts here. I really wish we could flag prose as ai-generated on here and just filter it out.

Drives me nuts that comments are held to a higher standard than submissions. HN is for conversation between humans[1] (about AI generated blogspam, apparently) 1. https://news.ycombinator.com/newsguidelines.html

Meta recently added a filter as a requirement for posts on Facebook. if it was ai generated, you are required to check off a box for that on your posts.

I've been asking for that for some time.

Re: Smaller, faster, safer: running Kimi and GLM at scale

#27

Post by maxmcd »

syntaxing wrote: post #11

> View pricing in the Cloudflare dashboard ↗ Why… I wanted to see if it’s worth it to use cloudflare’s endpoint but I can’t even see the pricing

I'm not sure how accurate this is, but there is pricing here: https://openrouter.ai/provider/cloudflare

Re: Smaller, faster, safer: running Kimi and GLM at scale

#28

Post by TacticalCoder »

brokenodo wrote: post #2

I was interested in reading this until my slop detector went off at the paragraph starting with “It's worth being precise about where the benefit comes from, because it isn't raw speed.” I love AI, but I really hate reading it.

> “It's worth being precise about where the benefit comes from, because it isn't raw speed.“

What's funny it's that is as if AI "learned" to speak english but not really. People simply don't speak using those strange constructs: those sentences sound a bit like if a "Karen" was trying to make a point.

What's scary, to me, as a dev using AI, is that those LLMs do the same thing with code: it looks like proper code, but it really ain't so once you dig a bit.

It's verbose and doesn't add anything: it's just infinite verbiage / sloppy-pasta.

Crazy thing though it's that it's 2026 and apparently devs can't be bothered to copy/paste their sloppy-pasta LLMish into a de-sloppifier before publishing blog posts.

Re: Smaller, faster, safer: running Kimi and GLM at scale

#29

Post by whimsicalism »

lostmsu wrote: post #20

So they quantize models, only tell about it in the blog post (instead of a warning on the model page), and even in the blog post pretend there's no difference by benchmarking on small context tasks many of which are saturated. Coding agents will probably be severely negatively affected by KV quantization. I'd say serving quantized models without saying so on the "store" page is fraud.

don't disagree, but there is a big difference between 'quantized model/weights' and quantized activations

Re: Smaller, faster, safer: running Kimi and GLM at scale

#30

Post by scrlk »

amluto wrote: post #16
scrlk wrote: post #4

Nice to see a provider being transparent about KV cache quantisation. I've been suspecting that some providers do this silently whilst heavily promoting their unquantised weights, even though KV quantisation can degrade quality more than weight quantisation. However, I wish their testing were more detailed. Firstly, some model families are more sensitive to KV quantisation than others (only Kimi K2.6 was tested). Sec…

They made an extremely strong claim: > None of this would matter if it changed the model's answers If they want to assert that the answers don’t change, then perhaps they should calculate the statistical distance between the token probability outputs or something to that effect. I doubt the results would indicate that the answers don’t change by any reasonable interpretation. Maybe the results are still good enough.

KL divergence is your friend when it comes to evaluating the effects of quantisation: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_diver...
Post reply on HN