Local LLM inference – impressive but too hard to work with • Hacker News Board

Re: Local LLM inference – impressive but too hard to work with

#61

Post by thomassmith65 » Wed, Apr 23, 2025, 2:44 AM UTC

Der_Einzige wrote: post #54

Why is HN so full of people who don't know good LLM tooling? SillyTavern and vllm is right there, ready to give you a class leading experience - but you all ignore it and use stuff like LM-studio (missing tons of features that SillyTavern or even oobabooga have, like advanced samplers such as min_p or top-nsigma) or worse you use even more slow solutions like ollama or llamacpp. The real reason that folks don't like…

It's the tone, not the content.

a) Slang signifies group membership, and can be irritating to a reader who does not identify with the group

b) In tech, there are too many knowledge domains for anyone to keep up. I personally find it frustrating when people expect everyone else to know about their area of expertise.

c) People have an urge to smack down someone who too loudly toots their own horn.

The final paragraph makes the tone problem worse; if we're being objective, it comes off as quite pissy, no? The same comment likely would have been upvoted if the tone had been gentler.

Re: Local LLM inference – impressive but too hard to work with

#62

Post by Der_Einzige » Wed, Apr 23, 2025, 3:15 AM UTC

thomassmith65 wrote: post #61

Der_Einzige wrote: post #54

Why is HN so full of people who don't know good LLM tooling? SillyTavern and vllm is right there, ready to give you a class leading experience - but you all ignore it and use stuff like LM-studio (missing tons of features that SillyTavern or even oobabooga have, like advanced samplers such as min_p or top-nsigma) or worse you use even more slow solutions like ollama or llamacpp. The real reason that folks don't like…

It's the tone, not the content. a) Slang signifies group membership, and can be irritating to a reader who does not identify with the group b) In tech, there are too many knowledge domains for anyone to keep up. I personally find it frustrating when people expect everyone else to know about their area of expertise. c) People have an urge to smack down someone who too loudly toots their own horn. The final paragraph m…

Strong opinions weakly held is a rallying cry here. Being spicy is how one drives engagement.

HN loves to act like they’re experts in LLMs but they’re mostly just not - and they don’t listen to their elders here.

Calling out poor tech choices with a “pissy” tone is simply continuing the tech culture of stack overflow, irc, etc. don’t act like it wasn’t awesome and you don’t like it.

If the person writing is too “good” at the tone side, it’s a sign that they spent countless time getting better at social skills which objectively trades off with time spent getting better at your field. Reality is a zero-sum game, time spent getting one skill is time spent not working on another. Often the best devs are extremely hard to work with and are spicy. Terry A Davis or Yoshua Bengio come to mind as examples here.

So, you should celebrate my tone. Not cry that your feelings were hurt by it. It means I did my homework so that you all don’t have to.

Re: Local LLM inference – impressive but too hard to work with

#63

Post by thomassmith65 » Wed, Apr 23, 2025, 3:39 AM UTC

Der_Einzige wrote: post #62

thomassmith65 wrote: post #61

Earlier quoted context omitted.

It's the tone, not the content. a) Slang signifies group membership, and can be irritating to a reader who does not identify with the group b) In tech, there are too many knowledge domains for anyone to keep up. I personally find it frustrating when people expect everyone else to know about their area of expertise. c) People have an urge to smack down someone who too loudly toots their own horn. The final paragraph m…

Strong opinions weakly held is a rallying cry here. Being spicy is how one drives engagement. HN loves to act like they’re experts in LLMs but they’re mostly just not - and they don’t listen to their elders here. Calling out poor tech choices with a “pissy” tone is simply continuing the tech culture of stack overflow, irc, etc. don’t act like it wasn’t awesome and you don’t like it. If the person writing is too “good…

The original comment seemed knowledgeable and informative.

It ended with a complaint about downvotes...

...but it began by insulting the audience.

It wasn't clear to me if you were too close to the comment to realize why people downvoted it.

I thought a critique might be helpful. Apparently, it wasn't welcome, but my intentions were good.