Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The L in "LLM" Stands for Lying

acko.net

Post reply on HN
Print view View original

111–120 of 500 posts

Re: The L in "LLM" Stands for Lying

#111

Post by stanko »

noemit wrote: post #106

Many people don't know this, but the Luddites were right. I studied Art History and this particular movement. One of the claims of the Luddites is that quality would go down, because their craft took half a lifetime to master (it was passed down from parent to chile.) I was able to feel wool scarves made in europe from the middle ages. (In museum storage, under the guidance of a curator) They are a fundamentally diff…

I think you are going to enjoy this talk by Jonathan Blow - Preventing the Collapse of Civilization:

https://www.youtube.com/watch?v=ZSRHeXYDLko

Re: The L in "LLM" Stands for Lying

#112

Post by mikkupikku »

dntrshnthngjxct wrote: post #77
tovej wrote: post #53

Earlier quoted context omitted.

An LLM has never saved me time. It has always produced something that doesn't quite work, has the rough shape of what I want, but somehow always gets all the details wrong. I can type up what I want much faster and be sure it's at least solving the right problem, even if it may have bugs. There are also tools to generate boilerplate that work much much better than LLMs. And they're deterministic.

If you do not plan out the architecture soundly, no amount of prompting will fix it if it is bad. I know this because my "handmade" project made with backward compatibility and horrible architecture keeps being badly fixed by LLM while the ones that rely on preemptive planning of the features and architecture, end up working right.

I think that's true, but something even more subtle is going on. The quality of the LLM output depends on how it was prompted in a way more profound than I think most people realize. If you prompt the LLM using jargon and lingo that indicate you are already well experienced with the domain space, the LLM will rollplay an experienced developer. If you prompt it like you're a clueless PHB who's never coded, the LLM will output shitty code to match the style of your prompt. This extends to architecture, if your prompts are written with a mature understanding of the architecture that should be used, the LLM will follow suit, but if not then the LLM will just slap together something that looks like it might work, but isn't well thought out.

Re: The L in "LLM" Stands for Lying

#113

Post by dannersy »

raincole wrote: post #34

> Video games stand out as one market where consumers have pushed back effectively No, it's simply untrue. Players only object against AI art assets. And only when they're painfully obvious. No one cares about how the code is written. If you actually read the words used in Steam AI survey you'll know Steam has completely caved in for AI-gen code as well. It's specifically worded like this: > content such as artwork,…

You're cherry picking. The open world games aren't as compelling anymore since the novelty is wearing off. I can cherry pick, too. For example, Starfield in all its grandeur is pretty boring.

And the users may not care about code directly, but they definitely do indirectly. The less optimized and more off-the-shelf solutions have seen a stark decrease in performance but allowing game development to be more approachable.

LLMs saving engineers and developers time is an unfounded claim because immediate results does not mean net positive. Actually, I'd argue that any software engineer worth their salt knows intimately that more immediate results is usually at the expense of long term sustainability.

Re: The L in "LLM" Stands for Lying

#114

Post by vntok »

Jooror wrote: post #74
vntok wrote: post #67

Earlier quoted context omitted.

> An LLM has never saved me time. It has always produced something that doesn't quite work, has the rough shape of what I want, but somehow always gets all the details wrong. This reads like a skill issue on your end, in part at least in the prompting side. It does take time to reach a point where you can prompt an LLM sufficiently well to get a correct answer in one shot, developing an intuitive understanding of wha…

I’m curious about how you landed “git gud; prompt better” and not “maybe the domain I work in is a better fit for LLM code”. Or, to be a bit less generous, consider the possibility that the code you’re generating is boilerplate, marshaling, and/or API calls. A facade of perceived complexity over something that’s as complex as a filter-map or two.

The parent is specifically talking about producing boilerplate code -a domain in which LLM excell at- and not having had any success at that. It's therefore not a leap of logic to assume they haven't put (enough) effort into getting better at prompting first, which is perfectly fine per se but leans towards a skill issue and not an immutable property of gen AI.

The uncomfortable fact remains that one cannot really expect to get much better results from an LLM without putting some work themselves. They aren't magical oracles.

Re: The L in "LLM" Stands for Lying

#115

Post by Nursie »

raincole wrote: post #34

> Video games stand out as one market where consumers have pushed back effectively No, it's simply untrue. Players only object against AI art assets. And only when they're painfully obvious. No one cares about how the code is written. If you actually read the words used in Steam AI survey you'll know Steam has completely caved in for AI-gen code as well. It's specifically worded like this: > content such as artwork,…

> Spore is well acclaimed.

Spore was fun (IMHO) but at the time of release was considered a disappointment compared to its hype.

Re: The L in "LLM" Stands for Lying

#116

Post by thunderbong »

artisin wrote: post #102
heavyset_go wrote: post #69

Earlier quoted context omitted.

Books are just simple theses and themes with hundreds of pages of boilerplate

Hit songs are just simple four-chord loops stretched over three minutes of synthetic boilerplate.

Both the books and the song analogies are incorrect. In the case of code, the users for whom the programmes are written, are not engaging with the statements of the code, they are interacting with interfaces the programmes provide.

This is not the same when it comes to books and music.

Re: The L in "LLM" Stands for Lying

#117

Post by mikkupikku »

Jooror wrote: post #74
vntok wrote: post #67

Earlier quoted context omitted.

> An LLM has never saved me time. It has always produced something that doesn't quite work, has the rough shape of what I want, but somehow always gets all the details wrong. This reads like a skill issue on your end, in part at least in the prompting side. It does take time to reach a point where you can prompt an LLM sufficiently well to get a correct answer in one shot, developing an intuitive understanding of wha…

I’m curious about how you landed “git gud; prompt better” and not “maybe the domain I work in is a better fit for LLM code”. Or, to be a bit less generous, consider the possibility that the code you’re generating is boilerplate, marshaling, and/or API calls. A facade of perceived complexity over something that’s as complex as a filter-map or two.

> I’m curious about how you landed “git gud; prompt better” and not “maybe the domain I work in is a better fit for LLM code”.

1. Personal experience. Lazy prompting vs careful prompting.

2. They're coincidentally good at things I'm good at, and shit at things I don't understand.

3. Following from 2, when used by somebody who does understand a problem space which I do not, they easily succeed. That dog vibe coding games succeeded in getting claude to write games because his master knew a thing or two about it. I on the other hand have no game Dev experience, even almost no hobby experience with games specifically, so I struggle to get any game code that even remotely works.

Re: The L in "LLM" Stands for Lying

#118

Post by sph »

stanko wrote: post #111
noemit wrote: post #106

Many people don't know this, but the Luddites were right. I studied Art History and this particular movement. One of the claims of the Luddites is that quality would go down, because their craft took half a lifetime to master (it was passed down from parent to chile.) I was able to feel wool scarves made in europe from the middle ages. (In museum storage, under the guidance of a curator) They are a fundamentally diff…

I think you are going to enjoy this talk by Jonathan Blow - Preventing the Collapse of Civilization: https://www.youtube.com/watch?v=ZSRHeXYDLko

I've had this talk in mind during the past 2/3 years of AI boom, and it feels like rewatching a video from the 80s about the dangers of global warming. Prescient, and perhaps a bit quaint in its optimism that somehow we won't make things even worse for ourselves.

Now we're way past the point of no return.

Re: The L in "LLM" Stands for Lying

#119

Post by Nursie »

doodaddy wrote: post #87

There’s a cold reality that we in this profession have yet to accept: nobody cares about our code. Nobody cares whether it’s pretty or clever or elegant. Sometimes, rarely, they care whether it’s maintainable. We are only craftsmen to ourselves and each other. To anyone else we are factory workers producing widgets to sell. Once we accept this then there is little surprise that the factory owners want us using a tool…

I accepted this years ago. In fact I go a step further - code is a liability.

It's certainly intellectually stimulating to create it, but I've learned to take joy in discarding vast swathes of it when it's no longer required.

Re: The L in "LLM" Stands for Lying

#120

Post by keyringlight »

theshrike79 wrote: post #41
raincole wrote: post #34

> Video games stand out as one market where consumers have pushed back effectively No, it's simply untrue. Players only object against AI art assets. And only when they're painfully obvious. No one cares about how the code is written. If you actually read the words used in Steam AI survey you'll know Steam has completely caved in for AI-gen code as well. It's specifically worded like this: > content such as artwork,…

Also "AI" has been in gaming, especially mobile gaming, for a literal decade already. Household name game studios have had custom AI art asset tooling for a long time that can create art quickly, using their specific style. AI is a tool and as Steve Jobs said, you can hold it wrong. It's like plastic surgery, you only notice the bad ones and object to them. An expert might detect the better jobs, but the regular folk…

Another example is upscaled texture mods, which has been a trend for a long while before 'large language' took off as a trend. Mods to improve textures in a game are definitely not new and that probably means including from other sources, but the ability to automate/industrialize that (and presumably a lot of training material available) meant there was a big wave of that mod category a few years back. My impression is that gamers will overlook a lot so long as it's 'free' or at least are very anti-business (even if the industry they enjoy relies upon it), the moment money is involved they suddenly care a lot about the whole fabric being hand made and need verification that everyone involved was handsomely rewarded.
Post reply on HN