Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Slopsquatting

en.wikipedia.org

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Slopsquatting

#31

Post by jen729w »

Invictus0 wrote: post #3

We need to have a professional software engineering license, at least for applications that are handling sensitive data. Why is it that it takes 1000 hours of study to cut people's hair, but anyone off the street can write some software that collects people's driver's licenses? (Looking at you, Tea app developers)

In 2023 I was at a talk at the National Press Club in Canberra, Australia, by the deputy head of one of our national intelligence services.

This was just after the Optus leak. Some hundreds of thousands of customers' data, down to the passport and DOB level, leaked. Again. I was going to ask him whether we, the collected IT consultants in the room, simply couldn't be trusted any more.

We've proven that we can't. I firmly believe that independent companies should no longer, by law, be able to collect my identifying information. If you must identify me, the state should provide a service. You hand off to them, they validate me, they send you a token back, I'm validated.

Sadly the microphone never made it to my corner of the room.

Re: Slopsquatting

#32

Post by iaiuse »

ysofunny wrote: post #23

all I want to figure out is how to "manually" (semi-manually) tweak the LLMs parameters so we can alter what it 'knows for sure' is this doable yet??? or is this one of those questions whose answer is best kept behind NDAs and other such practices?

Correct—they don’t “know” in the epistemic sense, but they do encode a latent world model that shows up as useful priors.

Put differently: GPT-4 isn’t a knowledge base, it’s a *Bayesian autocomplete* over dense vectors. That’s why it can draft Python faster than many juniors, yet fail a trivial chain-of-thought step if the token path diverges.

The trick in production is to sandwich it: retrieval (facts) LLM (fluency) rule checker (logic). Without that third guardrail, you’re betting on probability mass, not truth.

Re: Slopsquatting

#33

Post by aklemm »

zwnow wrote: post #29
ysofunny wrote: post #23

all I want to figure out is how to "manually" (semi-manually) tweak the LLMs parameters so we can alter what it 'knows for sure' is this doable yet??? or is this one of those questions whose answer is best kept behind NDAs and other such practices?

We still barely know how LLMs really work. Hard to tune things you dont understand. Inbefore people telling me "akshually we know all about bla bla bla..." no we dont.

It could be that we do know how they work, we just don’t know how to contextualize their output (as in, we ascribe way too much humanity to it) and that obscures what we think must be happening.

Re: Slopsquatting

#34

Post by glenstein »

>In May 2025, the potential and prevalence of slopsquatting was detailed in the academic paper "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs".[1][11] Some of the paper's main findings are that 19.7% of the LLM recommended packages did not exist

At the risk of perhaps misunderstanding or committing a category error, I wonder if there's such a thing as a category of "correct" hallucinating, distinct from things that are, in some sense, "known" via training (e.g. I read about prompting of one model showing it was able to accurately recreate most of the text of Harry Potter, so clearly it's "in there" somewhere).

An interesting upshot of that could be that models "grow" their own knowledge in an evolutionary way via hallucinations that are retained rather than pruned as part of routine filtering and training.

Though I'm sure some might suggest "hallucinating correctly" is just one of the same with ordinary b function. I wouldn't agree with that but I could at least see the argument.

Re: Slopsquatting

#35

Post by ysofunny »

zwnow wrote: post #29
ysofunny wrote: post #23

all I want to figure out is how to "manually" (semi-manually) tweak the LLMs parameters so we can alter what it 'knows for sure' is this doable yet??? or is this one of those questions whose answer is best kept behind NDAs and other such practices?

We still barely know how LLMs really work. Hard to tune things you dont understand. Inbefore people telling me "akshually we know all about bla bla bla..." no we dont.

which is why in order to appear smart and well informed, it's recommended to point out how my question is nonensense rather than legitimately trying to figure it out.

Re: Slopsquatting

#36

Post by Pxtl »

ikari_pl wrote: post #20
Pxtl wrote: post #14

Earlier quoted context omitted.

Case insensitivity is so important. Underscore is just capital hyphen.

I’d rather say underscore is a capital space :D

I'm tempted to make a keybind for this. Then I might actually start using snake_case instead of CamelCase, which on a certain level I know is better I just hate typing it.

Re: Slopsquatting

#37

Post by Hackbraten »

mapmeld wrote: post #22
kouteiheika wrote: post #6

> LLMs hallucinated a package named "huggingface-cli" [...] it is not the name of the package [...] software is correctly installed with [...] huggingface_hub It would be a good idea to disallow registering packages which only differ by '-'/'_'. Rust's crates.io does this, so if you register `foo-bar` you cannot register `foo_bar` anymore.

It is a command line tool "huggingface-cli", it's just installed with a differently-named pypi package. I wouldn't call this a full hallucination because anyone could make this mistake.

If a human user makes this mistake, they’d go to https://pypi.org/project/huggingface-cli/ and see that it a) either doesn’t exist, b) is an unrelated package, or c) that the verified list of its maintainers is unrelated than the maintainers on Hugging Face, Inc.’s GitHub repository.

Re: Slopsquatting

#38

Post by snapcaster »

Invictus0 wrote: post #15
snapcaster wrote: post #4

Earlier quoted context omitted.

I think something like professional licenses are easy to see benefits of but really hard to see the downsides. How many wonderful things _wont_ be created when you start gatekeeping something? Maybe it is worth it but it's not some free win

Like I said, the license should be for handling sensitive data. You're free to make doodle jump if you like.

I just don't like when comments on things like this don't engage with the downsides. Gatekeeping isn't "free" or strictly better

Re: Slopsquatting

#39

Post by fragmede »

EvanAnderson wrote: post #24
Invictus0 wrote: post #3

We need to have a professional software engineering license, at least for applications that are handling sensitive data. Why is it that it takes 1000 hours of study to cut people's hair, but anyone off the street can write some software that collects people's driver's licenses? (Looking at you, Tea app developers)

> Why is it that it takes 1000 hours of study to cut people's hair... Protectionism by a de facto trade guild was always my assumption. There are a lot of activities where bad practitioners present significant danger to society and licensure makes sense. I never understood how cutting hair rises to that level. I'd love to know how licensure in the barber profession is anything other than a bald-faced attempt at build…

> It seems like the market could correct for a bad practitioner in the barber space pretty easily,

Why do you assume that? I bet most people don't know their barber personally, and just go to the shop to get a cut. Should getting a haircut be fraught with having to go online and read a bunch of reviews, followed by the inevitable bickering between fake reviews and fake responses on top of that? No, I just want to get a decent cut for a decent price. We can nitpick over how much training is reasonable, and sure there's an element of protectionism there, but if the Internet had taught us anything, it's that online reviews are bullshit. I would hate to have to rely on them to correct for a bad practitioner when they aren't really able to do anything about bad doctors, which has a much higher bar to practice.

Post reply on HN