Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nano-vLLM: How a vLLM-style inference engine works

neutree.ai

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Nano-vLLM: How a vLLM-style inference engine works

#21

Post by selfsimilar »

Juvination wrote: post #13
_alternator_ wrote: post #11

Earlier quoted context omitted.

Wait—do people here really think the em dash was nonexistent before LLMs? It’s widely used by people like me who care about writing style. The reason LLMs use it is because they reflect care and concern about writing style.

Nobody ever said that they were nonexistent before LLMs. When you are investigating and trying to determine if something is AI generated they are the number one indicator. So if you're being accused of just spewing AI, then double down and spew what looks EVEN MORE like AI. What are you even doing?

Number one indicator? A single punctuation mark that's trivial to make on most keyboards (option-dash on macOS). And generally people who write software are extra fixated on punctuation for obvious reasons: missing semi-colons break your build, etc. Maybe in some other niche message board people will use dash and em dash interchangeably, but here?

Also, if the a single character is how you're red-flagging LLM output, do you know how easy it is to avoid? I didn't use it here at all, but how do you know I didn't run this through some slop-machine to tighten my prose? It's really low-effort take to say "just avoid em dashes so we know you're not an AI".

https://www.mcsweeneys.net/articles/the-em-dash-responds-to-...

Re: Nano-vLLM: How a vLLM-style inference engine works

#23

Post by tubs »

yz-yu wrote: post #20
CodeMage wrote: post #14

Earlier quoted context omitted.

It does, but what does that say about the state of communication in our industry? I've seen a lot of writing that reads like an AI produced it in contexts where I could be pretty sure no AI was involved. We want to sound professional, so we sanitize how we write so much that it becomes... whatever this current situation is. No offense intended to @yz-yu, by the way. I miss the times when more people wrote in an eccen…

The comments here turned out much more interesting than I expected—this has become a great place to discuss the difference between AI-generated, AI-written, and AI-assisted content. So let me start from @jbarrow's comment: "AI written, generated from the codebase." My actual learning process looked like this: 1. I walked through the nano-vLLM codebase, asking Claude Code some high-level questions to warm up. 2. Then…

When text is (clearly) non native English I think most native readers don’t even register grammar errors.

To be honest most native readers wouldn’t register grammar errors full stop.

I guess I have more awe of people who speak a foreign language at all compared to piping it through some agent malarkey.

Re: Nano-vLLM: How a vLLM-style inference engine works

#25

Post by pessimizer »

selfsimilar wrote: post #21
Juvination wrote: post #13

Earlier quoted context omitted.

Nobody ever said that they were nonexistent before LLMs. When you are investigating and trying to determine if something is AI generated they are the number one indicator. So if you're being accused of just spewing AI, then double down and spew what looks EVEN MORE like AI. What are you even doing?

Number one indicator? A single punctuation mark that's trivial to make on most keyboards (option-dash on macOS). And generally people who write software are extra fixated on punctuation for obvious reasons: missing semi-colons break your build, etc. Maybe in some other niche message board people will use dash and em dash interchangeably, but here? Also, if the a single character is how you're red-flagging LLM output,…

Yes, number one indicator. Yes, of course you can go through the output and take out all of the em-dashes. Then the number one indicator will obviously not work.

Re: Nano-vLLM: How a vLLM-style inference engine works

#26

Post by jacquesm »

lambda wrote: post #18
jacquesm wrote: post #5

Earlier quoted context omitted.

Funny, this reads even more AI written than the article itself.

One thing to keep in mind is that a lot of non-native English speakers use LLMs to translate to English, or to polish their English prose; they may not realize that it causes the translation to come out in a very LLM-style tone. Not sure if that's the case here, but it looks like OP is a native Chinese speaker so may be using tools to translate to English.

It looks like you were right about that.

https://news.ycombinator.com/item?id=46858409

But: this was never a problem and now we have to distinguish between LLM generated, human generated, LLM polished and human generated. I'd much prefer it if people just wrote their own text, warts and all.

Re: Nano-vLLM: How a vLLM-style inference engine works

#29

Post by moi2388 »

_alternator_ wrote: post #11
Juvination wrote: post #9

Earlier quoted context omitted.

The em dashes really aren't helping their case.

Wait—do people here really think the em dash was nonexistent before LLMs? It’s widely used by people like me who care about writing style. The reason LLMs use it is because they reflect care and concern about writing style.

Not non existent, but rare. And again the presumption was correct, the text was put through an LLM.

Re: Nano-vLLM: How a vLLM-style inference engine works

#30

Post by lambda »

yz-yu wrote: post #20
CodeMage wrote: post #14

Earlier quoted context omitted.

It does, but what does that say about the state of communication in our industry? I've seen a lot of writing that reads like an AI produced it in contexts where I could be pretty sure no AI was involved. We want to sound professional, so we sanitize how we write so much that it becomes... whatever this current situation is. No offense intended to @yz-yu, by the way. I miss the times when more people wrote in an eccen…

The comments here turned out much more interesting than I expected—this has become a great place to discuss the difference between AI-generated, AI-written, and AI-assisted content. So let me start from @jbarrow's comment: "AI written, generated from the codebase." My actual learning process looked like this: 1. I walked through the nano-vLLM codebase, asking Claude Code some high-level questions to warm up. 2. Then…

> I wrote this comment the same way. The LLM fixed 14 grammar mistakes that I think would distract readers more than any LLM-ish phrasing.

I don't think that assumption is correct. As you can see by the discussion we're having here, the LLM "fixed" text is actually quite distracting, while text written by a reasonably proficient non-native speaker is generally perfectly readable. It's only if your English is extremely poor to non-existant that it makes more sense to use machine translation or editing rather than writing it yourself.

One problem is that people are becoming quite sensitive to slop, where people just post completely unreviewed, AI generated text. It's quite frustrating, because it's asking readers to read something that no one has ever bothered to write, and it frequently crowds out discussion that people are more interested in. So everyone is kind of hyper-sensitive to signs of AI written text right now, which means when you start to see such signs, your brain moves over to trying to interpret whether it's AI generated rather than reading the text itself.

Post reply on HN