Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't credit the LLM

isaacsu.com

Post reply on HN
Print view View original

11–20 of 51 posts

Re: Don't credit the LLM

#11

Post by voidhorse »

eru wrote: post #6
NitpickLawyer wrote: post #5

> Why did I need to know that? In a perfect world such a disclaimer would help anyone reading the output by setting expectations. We each have different experiences with this, and different levels of acceptance. Knowing that something is using generated content can help anyone either accept / skim / thoroughly review something. The problem is that we're seeing lots of backlash, often for ideological reasons, or for o…

Agreed. No one minds when I credit a fuzzer for finding a bug. I hope we can use the same pragmatic attitude for LLM help.

Pragmatically, the range of application for a fuzzer and for an LLM are so widely different that I'm not sure it makes any kind of pragmatic sense to compare the two.

Also, I don't hear a lot of people credit fuzzers. I do hear a lot of people say things like "we are fuzzing the code"'just like they say "we are hammering the nails" and not "wow hammer great job getting that nail in place"

Re: Don't credit the LLM

#12

Post by nlawalker »

It’s #3, it’s a disclaimer, you’re helpful if it’s right and not to blame if it’s wrong.

When I see it, it always makes me want to respond “weird, I asked Claude too and it said ”.

Re: Don't credit the LLM

#13

Post by chaidhat »

I agree but not for the reasons the article states. If you say “an LLM made this” and are proven to be wrong, then you can divert blame to the LLM: this is bad as you don’t in the future try to improve yourself and verify the LLM.

Re: Don't credit the LLM

#14

Post by jimmaswell »

Driver4732 wrote: post #9

I'm always skeptical of a recommendation to a technical problem when someone tells me they used an LLM chatbot like CoPilot. They really are just slot machines. Sometimes the answer works. Sometimes it doesn't. On the other hand, when someone tells me they googled it and found an article - not that AI overview crap - I trust it more because I know it was written and tested by a human.

I find human answers and handwritten code to be much worse of a slot machine. An article (or stack overflow post) has a not-insignificant chance to be riddled with outdated API calls, human-made bugs, unspoken assumptions of knowledge on my part or how the environment it runs in will be set up, etc. Meanwhile a frontier model easily deals with all that for you and gives you something that almost always just works, tailored to your exact requirements, with thorough test cases and validation that it works before handing it off to you. By the time you've hit roadblock number 4 out of 20 trying to adapt that human slop to fizzbar 2.0 because 1.9 was deprecated, the LLM would have been done.

Re: Don't credit the LLM

#15

Post by txrx0000 »

You compare LLMs to spellcheckers, but they are clearly very different in capability at this point. These agents are becoming more human-like and "just a tool" is no longer a fair characterization.

And even if you believe it's just a tool, a text generator that takes in and spits out the Internet, you should cite it like citing a website.

Re: Don't credit the LLM

#16

Post by zkmon »

Personification an LLM has become a style or trend. Saying "Sol solved this problem for me" sounds great, indicating your honesty in giving credit to LLM. However this is wrong. You are "using" LLM to solve something, just like "using" a calculator to do some computation. Still the computation is done by you, not calculator. When you drove from X to Y, you did it, not the car.

Re: Don't credit the LLM

#18

Post by leonidasv »

nlawalker wrote: post #12

It’s #3, it’s a disclaimer, you’re helpful if it’s right and not to blame if it’s wrong. When I see it, it always makes me want to respond “weird, I asked Claude too and it said ”.

I don't see like that, for me it's like: here's an approximate answer. Is that sufficient or do you need a 100% accurate number? I can get this info for you, but I will have to stop everything I'm doing right now to figure it out or do it after I'm done with other priorities.

Sometimes the person asking just needs an estimate, a ballpark number. It doesn't really make a difference if 17 or 18 support requests were received.

Re: Don't credit the LLM

#19

Post by cush »

nlawalker wrote: post #12

It’s #3, it’s a disclaimer, you’re helpful if it’s right and not to blame if it’s wrong. When I see it, it always makes me want to respond “weird, I asked Claude too and it said ”.

That’s important to know too though. Most people are still figuring out this new world and knowing where it breaks it pretty critical right now.

Re: Don't credit the LLM

#20

Post by voidhorse »

grayclhn wrote: post #7

This is a little silly — if I quote a large passage from an encyclopedia (or whatever), I cite the source. If I use an LLM to write some of a paper or report, I’m not writing that part (even if I edit it or rephrase it or whatever) so I’d better credit the LLM in whatever way is most consistent with how its output is being presented in the paper.

I think there is a clear distinction to make in the case of communications.

Yes, if you are writing a longer piece of communication and you just verbatim ship whatever the LLM spits out, it is important that you note that in the communication. This obviously applies to deepfakes as well.

But this quickly gets slippery. If an LLM generates an initial draft of a paragraph, and I edit it 10 times making mostly stylistic changes, do I need to cite the LLM for that passage? How much personal touch is enough to absolve one of LLM citation? We don't require filmmakers to present a "this is CGI" banner during films (only occasionally in trailers), if a filmmaker uses an LLM to generate 20% of the contents of a film, much in the manner of CGi usage, do they need to add a disclaimer?

The text produced by an LLM really doesn't function the same way text produced by humans does. The cited human is a stable entity that exists, not a stateless machine. You can ask them about their quoted work, look up their history, etc. There is a stable participant in the field with a stable identity that you can talk to. Not so for LLM produced text.

At the end of the day whatever it is your present is coming from you. Own it. If you feel the need to hedge that such and such was from an LLM it probably means that it's bullshit that you'd better rework or at least understand much better before you pass your slop off to some poor coworker

Post reply on HN