Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity has left the chat: The price of debiasing language models

arxiv.org

Post reply on HN
Print view View original

141–150 of 238 posts

Re: Creativity has left the chat: The price of debiasing language models

#141

Post by inglor_cz »

Barrin92 wrote: post #70
lukan wrote: post #58

Earlier quoted context omitted.

"(also, categorizing propaganda posters as art, ewwh...)" Heinrich Heine, the german Poet declined working for the socialist party despite symphatising saying something like: I want to remain a poet, you want a propagandist. A poet cannot be a propagandist at the same time.

Art for much of human history was devotional, a lot of our greatest artworks today are still religious in nature. The idea that art solely is an act of rebellion rather than say worship, is a pretty modern idea that has produced some rather questionable art by the way. Of course a great artist or poet can be a propagandist. Riefenstahl, Mann, a lot of German nationalists were great artists. One of the most famous wor…

There is a difference between devotional art inspired by religion / mythical events long past (Aeneid etc.), and between the sort of propaganda that the modern totalitarian state demands, which usually centers around some living or freshly dead leader.

I'd be open to discussion where the exact limit is. Lenin died in the 1920s, Marx even earlier, but those two were frequently depicted in Communist propaganda of the 1980s.

So it is probably "hundreds of years".

Re: Creativity has left the chat: The price of debiasing language models

#142

Post by inglor_cz »

b800h wrote: post #9

Well this is just like humans. Totalitarian societies don't produce great creative work. I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in. At some point we'll get to the electronic equivalent of Winston Smith with the rats.

"Totalitarian societies don't produce great creative work."

You contradict yourself a bit - Havel did produce his work while living in a totalitarian country.

I would say that government-supported art is rarely creative even in democratic countries, and the more totalitarian the government, the less creative official art.

But as long as the goverment gives the society some space to breathe and squeeze creative instincts through, some of the artists will attempt to circumvent the official taboos and create outstanding work, even if it is suppressed later when the times get tougher.

Czechoslovakia in the 1960s to 1980s produced a lot of great creative work, even though a lot of it was banned either immediately or after the Soviet invasion of 1968.

The same countries (CZ and SK) as democracies are remarkably less creative. Once there is no monster to fight against, artists become bored or too self-absorbed to be understandable to the common folks.

Re: Creativity has left the chat: The price of debiasing language models

#143

Post by fsmv »

MrThoughtful wrote: post #8

How hard would it be to create a "raw" model on a corpus like Hacker News or Wikipedia? With "raw", I mean that it is simply trained to predict the next token and nothing else. Would be fun to play with such a model.

There are some that exist. The problem is you need at least some RLHF to make it follow instructions instead of just predicting sentences.

Re: Creativity has left the chat: The price of debiasing language models

#144

Post by dang »

ziggy_star wrote: post #140
shrimp_emoji wrote: post #139

Earlier quoted context omitted.

Shine on, you crazy diamond.

Thanks, I love the many versions of this song, fun to jam :) BTW this is how these exchanges looks like in my browser: https://imgur.com/qLwS1jV Hi Dang! What's up with all that? I emailed and everything.

Users are flagging your posts. We can only guess why users flag things, but I don't think it's hard to guess in this case: you're posting too many low quality comments, especially unsubstantive comments and/or flamebait. Could you please stop doing that? It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Re: Creativity has left the chat: The price of debiasing language models

#146

Post by klyrs »

knallfrosch wrote: post #79
jeroenvlek wrote: post #12

Earlier quoted context omitted.

Still anecdotal, but I can only confirm this with my own experience. The worst was when I was debugging code, described the problem to GPT-4o, and then got my exact same code back with some blanket statements like "print your output for debugging" etc. This happened a couple of times over separate chats.

gpt-4 has had serious laziness problems for over a year now. It keeps on telling me, what I should and could do, instead of doing it itself.

The irony here is incredible. The LLM is lazy, you say? I do wonder where it learned that...

Re: Creativity has left the chat: The price of debiasing language models

#147

Post by core-e »

knallfrosch wrote: post #79
jeroenvlek wrote: post #12

Earlier quoted context omitted.

Still anecdotal, but I can only confirm this with my own experience. The worst was when I was debugging code, described the problem to GPT-4o, and then got my exact same code back with some blanket statements like "print your output for debugging" etc. This happened a couple of times over separate chats.

gpt-4 has had serious laziness problems for over a year now. It keeps on telling me, what I should and could do, instead of doing it itself.

In other words it's giving more human like responses...

Re: Creativity has left the chat: The price of debiasing language models

#148

Post by SirMaster »

I feel like "information systems" have always struggled with bias, and the latest AI/ML systems seem to be no different.

It doesn't really seem like a problem that can or will ever be "solved". Just mitigated to various extents, but there will still likely be some underlying biases that exist that are not fully or effectively filtered. Because to adjust a bias seems to mean you have to detect and understand it first.

It feels like it would be a full-time job to keep making sure some evolving model continued to stay "neutral".

Re: Creativity has left the chat: The price of debiasing language models

#149

Post by mdp2021 »

mdp2021 wrote: post #127
ziggy_star wrote: post #124

Earlier quoted context omitted.

[flagged]

> I shall endeavor "Appropriate" will be enough, and to the point ;) > keeping schtum Oh that is a vice you will find frequently here - reacting but not expressing, condemning but nor orienting...

> Oh that is a vice you will find frequently here

↑

See?

Re: Creativity has left the chat: The price of debiasing language models

#150

Post by wongarsu »

flohofwoe wrote: post #53
rhcom2 wrote: post #22

Earlier quoted context omitted.

> Authoritarian societies don't produce great creative work. Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.

Eastern European science fiction would be a better example. Authors like Stanislaw Lem or the Strugatski brothers had to adapt to sneak critical ideas past censors, and readers had to adapt and read between the lines. (also, categorizing propaganda posters as art, ewwh...)

Soviet bus stops are another great example. Most Soviet architecture was forced to be very utilitarian, but bus stops managed to be the rare exception and thus got a lot of creative energy

https://www.wired.com/2015/09/wild-architecture-soviet-era-b...

https://www.boredpanda.com/most-peculiar-soviet-bus-stops-ch...

Post reply on HN