Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Antislop: A framework for eliminating repetitive patterns in language models

arxiv.org

Post reply on HN
Print view View original

101–110 of 119 posts

Re: Antislop: A framework for eliminating repetitive patterns in language models

#101

Post by vintermann »

topaz0 wrote: post #79
rogerkirkness wrote: post #44

Earlier quoted context omitted.

This is the main thing that immediately tells me something is AI. This form of reasoning was much less common before ChatGPT.

I don't think this is true. The LLMs use this construction noticeably more frequently than normal people, and I too feel the annoyance when they do, but if you look around I think you'll find it's pretty common in many registers of human natural english.

And each of us has patterns. I bet if you read a million of my posts, you would be annoyed with my writing idiosyncrasies too.

Re: Antislop: A framework for eliminating repetitive patterns in language models

#102

Post by vintermann »

SoftTalker wrote: post #21

I honestly can’t always distinguish AI slop from the formulaic corp-speak used in emails and memos and brochure websites and other marketing. I’m guessing that must be a large component of the training matter.

I don't think that's a coincidence. Right now a lot of the business proposition for LLM bots is selling it to corporations as the ultimate corporate yes-man.

Re: Antislop: A framework for eliminating repetitive patterns in language models

#103

Post by EigenLord »

Interesting work but this strikes me as a somewhat quixotic fight against inevitable tendencies of statistical models. Reinforcement learning has a single goal, an agreeable mean. Reinforcement learning stops when the LLM produces agreeable responses more often than not, the only way you can achieve absolute certainty here is if you tune it for an infinite amount of time. I also don't see how this method couldn't be subsumed by a simpler method like dynamic temperature adjustment. Transformers are fully capable of generating unpredictable yet semantic text based on a single hyperparameter. Maybe it would make more sense to simply experiment with different temperature settings. Usually it's a fixed value.

Re: Antislop: A framework for eliminating repetitive patterns in language models

#104

Post by BiteCode_dev »

atourgates wrote: post #15

I've been using ChatGPT fairly regularly for about a year. Mostly as an editor/brainstorming-partner/copy-reviewer. Lots of things have changed in that year, but the things that haven't are: * So, so many em-dashes. All over the place. (I've tried various ways to get it to stop. None of them have worked long term). * Random emojis. * Affirmations at the start of messages. ("That's a great idea!") With a brief pause w…

I think it's the default behavior, because it's cheaper and faster to produce than the real answer.

I assume the beginning of the answer is given to a cheaper, faster model, so that the slower, more expensive one can have time to think.

It keeps the conversation lively and natural for most people.

Would be interesting to test if it's true, by disabling it with a system prompt, and measure if the time-to-answer is slower for the first word.

Re: Antislop: A framework for eliminating repetitive patterns in language models

#105

Post by JimDabell »

WASDx wrote: post #55
atourgates wrote: post #15

I've been using ChatGPT fairly regularly for about a year. Mostly as an editor/brainstorming-partner/copy-reviewer. Lots of things have changed in that year, but the things that haven't are: * So, so many em-dashes. All over the place. (I've tried various ways to get it to stop. None of them have worked long term). * Random emojis. * Affirmations at the start of messages. ("That's a great idea!") With a brief pause w…

You can customize it to get rid of all that. I set it to the "Robot" personality and a custom instruction to "No fluff and politeness. Be short and get straight to the point. Don't overuse bold font for emphasis."

If I tell it no fluff, the only thing that changes is that it starts out with responses like “Sure, here’s what you asked for with no fluff…”.

Re: Antislop: A framework for eliminating repetitive patterns in language models

#106

Post by BAPHOMETA88F »

Parsing a LLM as the measure of a series of qc metrics, which isolate for preference strings, whether lexical weights or parameters. Can this create the rules for formal understanding or correlating libraries of Babel?

Searle's paper calls these questions, script, or a story.

[1]:https://web.archive.org/web/20071210043312/http://members.ao...

Re: Antislop: A framework for eliminating repetitive patterns in language models

#107

Post by Starlevel004 »

razodactyl wrote: post #90
bakugo wrote: post #42

Earlier quoted context omitted.

Don't forget the classic: "It's not just X—it's Y."

Absolutely this. I feel like I'm having an immune response to my own language. These patterns irk me in a weird way. Lack of variance is jarring perhaps? Everyone sounding more robotic than usual? Mode-collapse of normal language.

It sounds like LinkedIn speak which most people have a natural immune reaction to.

Re: Antislop: A framework for eliminating repetitive patterns in language models

#108

Post by mnky9800n »

totetsu wrote: post #85
estimator7292 wrote: post #17

Earlier quoted context omitted.

Ah, you've hit a classic problem with :smile_with_sweat_drop:. Your intuition is right-- but let me clarify some subtleties...

Yeah, that’s a really insightful point, and you’ve kind of hit the nail on the head…

yesterday it told me the "juice wasn't worth the squeeze."

Re: Antislop: A framework for eliminating repetitive patterns in language models

#109

Post by aeonik »

totetsu wrote: post #85
estimator7292 wrote: post #17

Earlier quoted context omitted.

Ah, you've hit a classic problem with :smile_with_sweat_drop:. Your intuition is right-- but let me clarify some subtleties...

Yeah, that’s a really insightful point, and you’ve kind of hit the nail on the head…

You didn't just give a compliment, you forged a symbolic bridge between islands of meaning!
Post reply on HN