Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Generative AI Con

wheresyoured.at

Post reply on HN
Print view View original

191–200 of 503 posts

Re: The Generative AI Con

#191

Post by simonw »

Lio wrote: post #131
simonw wrote: post #5

> When you put aside the hype and anecdotes, generative AI has languished in the same place, even in my kindest estimations, for several months, though it's really been years. The one "big thing" that they've been able to do is to use "reasoning" to make the Large Language Models "think" [...] This is missing the most interesting changes in generative AI space over the last 18 months: - Multi-modal: LLMs can consume…

> LLMs can consume images, Not very well in my experience. Last time I checked ChatGPT/DALL-E couldn't understand the its own output to know that what it had drawn was incorrect. Nor could it correct mistakes that were pointed out to it. For example, I ask it to draw an image of a bike with rim brakes it could not, nor could it "see" that what was wrong with the brakes that it had drawn. For all intents and purposes…

Generating images and consuming images are very different challenges, which for most models use entirely different systems (ChatGPT constructs prompts to DALL-E for example: https://simonwillison.net/2023/Oct/26/add-a-walrus/ )

Evaluating vision LLMs on their ability to improve their own generation of images doesn't make sense to me. That's why I enjoy torturing new models with my pelican on a bicycle SVG benchmark!

Re: The Generative AI Con

#192

Post by linuxftw »

stpedgwdgfhgdd wrote: post #171

Software development to me has always been about the 80-20 rule. You build 80% of the functionality in 20% of the time. Next you spent 80% of your time to build the remaining 20%. With LLMs it feels we are getting near to 90-10. Finding the bug in those good-looking pieces of generated code is pretty hard. (After all, you did not pay a lot of attention to the generated code, it looked pretty solid) Some will argue th…

I feel like finding bugs in golang is a non issue. The language is typed, so obvious problems are caught early in the editor. Unit tests mop up the rest (unit tests which are also written by copilot in my case).

How I write unit tests: Open the chat menu, paste in function signature, describe the tests I want. Out pop the tests. Run, fix code as needed. Add more tests, etc. Super easy.

Re: The Generative AI Con

#193

Post by magicalist »

advael wrote: post #96
xnx wrote: post #54

Earlier quoted context omitted.

For better or worse, "LLM" or "generative ai" has become roughly synonymous with the current wave of ML. I know very little about ChatGPT, but Waymo is using an LLM: "Powered by Gemini, a multimodal large language model developed by Google, EMMA employs a unified, end-to-end trained model to generate future trajectories for autonomous vehicles directly from sensor data." ( https://waymo.com/blog/2024/10/introducing-e…

Huh. I mean it makes sense to train end-to-end on all the interrelated tasks involved in driving but putting a whole-ass language model in the middle of that seems like a stunt. I wonder if it does better than like, any random transformer not trained on language first? Still, I hadn't heard that so I guess I was wrong about that one

No, you were right, this appears to be just research on how applicable LLMs could be to the space. They talk about the improvements their LLM makes, especially in being multimodal vs training multiple independent models, but also the limitations that appear to prevent it from being useable as it is. Maybe some form if it will be used some day (it does seem like it would be useful to have semantic understanding of the world integrated into the system), but at least as of when this was published, it's not actually used.

Re: The Generative AI Con

#194

Post by suraci »

simonw wrote: post #177
suraci wrote: post #78

Earlier quoted context omitted.

> If nothing else, my workflows as a software developer have changed significantly in these past two years with just what's available today, and there is so much work going into making that workflow far more productive. this is exactly the problem The more productivity AI brings to workers, the fewer employees employers need to hire, the less salary employers need to pay, and the less money workers have for consumpti…

What's your opinion on the productivity boost open source libraries have brought to developers? Did all of that free code reduce demand for developers? If not, why not?

> Did all of that free code reduce demand for developers?

the anwser is yes, while in the meantime, the expansion of the industry offset the surplus of developers.

Re: The Generative AI Con

#195

Post by camdenreslink »

qingcharles wrote: post #79
simonw wrote: post #46

Earlier quoted context omitted.

Multi-modal audio is great. I talk to ChatGPT when I'm cooking or walking the dog. For images I use it for things like helping draft initial alt text for images, extracting tables from screenshots, translating photos of signs in languages I don't speak - and then really fun stuff like "invent a recipe to recreate this plate of food" or "my CSS renders like this, what should I change?" or "How do you think I turn on t…

Multi-modal was the absolute game-changer. Just last night I was digging around in my basement, pulling apart my furnace, showing pics of the inside of it, having GPT explain how it works and what I needed to do to fix it.

I would never trust an LLM to do this unless it was pointing me to pages/sections in a real manual or reputable source I could reference.

Re: The Generative AI Con

#196

Post by Retr0id »

nbuujocjut wrote: post #178

GitHub Copilot and similar tools make good developers more productive. This alone is a genuine use case with some associated value. Is it enough to justify the valuations of OpenAI etc? Probably not by itself. But I expect other industries have similar productivity boosts where people learn to use the tools appropriately. What’s the total business opportunity of making all knowledge workers 10% more productive (to pi…

> GitHub Copilot and similar tools make good developers more productive.

Do we have any empirical evidence of this? It seems like it'd be an easy experiment to run - task a number of teams with building a particular product, some with Copilot and some without, and see what happens.

I've tried copilot myself, and at times it makes me feel more productive, but I can't tell if it's truly helping me overall.

Re: The Generative AI Con

#197

Post by Red_Comet_88 »

mitthrowaway2 wrote: post #28
lukevp wrote: post #12

I’m a little shocked at how much negativity there is around LLMs among developers. It’s a new tool that requires some learning, and it’s sometimes not so great, but if you’ve used an IDE with real coding assistance built in (eg. VS Code in Edit with Copilot mode - NOT Chat mode, using Claude 3.5), it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10…

LLMs are pretty good at the aspects of coding that I consider to be "the fun part". Using them has made me more productive, but also made my job less fun, because I can't justify spending time using my own brain to do "the fun part" on my employer's dime. And that was something I was particularly good at, which is why I was able to be paid well to do it. So now my company makes more money, and the work gets done fast…

Bit of a boomer statement here but maybe this will encourage devs such as yourself to contribute more to open source passion projects that will help dethrone the monopolies. Looking at Valve's investment into Linux via Proton as a great example.

It would be so nice to have a productivity Linux OS that just works on all my devices without tinkering. I want to stop supporting the closed source monopolies, but the alternatives aren't up to par yet. I am extremely hopeful that they will be once mega corps inevitably decay and people tire of the boom-bust cycle.

As technologists, we all want beautifully designed tools, and I'm increasingly seeing that these are only created by passionate and talented people who truly care about tech, unlike megacorps that only care about enriching their board and elite shareholders.

Re: The Generative AI Con

#198

Post by suraci »

andsoitis wrote: post #174
suraci wrote: post #78

Earlier quoted context omitted.

> If nothing else, my workflows as a software developer have changed significantly in these past two years with just what's available today, and there is so much work going into making that workflow far more productive. this is exactly the problem The more productivity AI brings to workers, the fewer employees employers need to hire, the less salary employers need to pay, and the less money workers have for consumpti…

Less productivity seems like a worse path.

it depends how you define good or worse

for humanity, the increase in productivity is progress

i'm not saying it's bad, i'm saying it has consequences

Re: The Generative AI Con

#199

Post by sandos »

nbuujocjut wrote: post #178

GitHub Copilot and similar tools make good developers more productive. This alone is a genuine use case with some associated value. Is it enough to justify the valuations of OpenAI etc? Probably not by itself. But I expect other industries have similar productivity boosts where people learn to use the tools appropriately. What’s the total business opportunity of making all knowledge workers 10% more productive (to pi…

Generative AI only really helps (a meaningful amount) developers where writing code is actually the bottleneck. I have not been in a such a position for years and years.

Re: The Generative AI Con

#200

Post by jcgrillo »

simonw wrote: post #185
jcgrillo wrote: post #140

Earlier quoted context omitted.

> Cost as in, cost to you? Or cost to serve? This. IIUC to serve an LLM is to perform an O(n^2) computation on the model weights for every single character of user input. These models are 40+GB so that means I need to provision about 40GB RAM per concurrent user and perform hundreds of TB worth of computations per query. How much would I have to charge for this? Are there any products where the users would actually g…

The efficiency gains over the past 18 months have been incredible. Turns out there was a lot of low hanging fruit to make these things faster, cheaper and more resource efficient. https://simonwillison.net/2024/Dec/31/llms-in-2024/#llm-pric...

Interesting. There's obviously been a precipitous drop in the sticker price, but has there really been a concomitant efficiency increase? It's hard to believe the sticker price these companies are charging has anything to do with reality given how massively they're subsidized (free Azure compute, billions upon billions in cash, etc). Is this efficiency trend real? Do you know of any data demonstrating it?
Post reply on HN