Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Artificial intelligence is losing hype

economist.com

Post reply on HN
Print view View original

351–360 of 814 posts

Re: Artificial intelligence is losing hype

#351

Post by yoyohello13 »

julienchastang wrote: post #240

As usual, when we see a thread on this topic on HN, the reactions tend to be bimodal: either "Yes, AI has transformed my workflow" (which is where I mostly fall), or "No, it's over-hyped." The latter often comes with an anecdote about how an LLM failed at a relatively simple task. I speculate that this diversity in opinion might be related to whether or not the user is employing a pro-tier LLM. Personally, I've been…

I’m in the middle. I use paid models. It’s great for scripts, reminding me of commands, and surprisingly good at debugging error messages. But if I’m working in our main code base I’m basically never able to just use the code it spits out. It can get me in the ballpark of what I want, but it often does weird convoluted shit on the first pass. I can keep prompting to refine the code down to something closer to what I want, but at that point it would have just been faster to write the thing myself.

I kind of like LLMs for learning new languages. Claude or ChatGPT are good for asking questions. But copilot really stunts learning for me. I find nothing really sticks in my brain when I have copilot running. I feel like I just turn my brain off, which seems kind of dangerous to me in the long run.

Re: Artificial intelligence is losing hype

#352

Post by yoyohello13 »

anon291 wrote: post #340
codexon wrote: post #296

Earlier quoted context omitted.

I'm using the latest top paid models, gpt4o and claude 3.5. They work when there's a lot of examples on github or google, but once you get into something that doesn't have a lot of examples like closed source code or rarely used libraries, it will start hallucinating and even mixing up different API versions to create a mess that doesn't work at all. I don't believe LLMs will get any better than this without a new ma…

> don't believe LLMs will get any better than this without a new major breakthrough, but this is already better than using Google search I mean .. every single org is invested in that research right now

Money helps, but doesn’t guarantee breakthroughs.

Re: Artificial intelligence is losing hype

#353

Post by iluvcommunism »

wokwokwok wrote: post #347
_acco wrote: post #294

Earlier quoted context omitted.

Context is the most challenging bit. FWIW, the codebases I'm working on are still small enough to where I rarely need to include more than 12 files into context. And I find as I make the context bigger beyond that, results degrade significantly. So I don't know how this would go in a much larger codebase. What floored him was simply how much of my programming I was doing with an LLM / how little I write line-by-line…

So you spend 10 minutes writing a free text description of the test you want; tell it exactly how you want it to write the test, and then 4-5 minutes trying to understand if it did the right thing or not, restart because it did something crazy then a few minutes manually fixing the diff it generated? MMmm. I mean, don't get me wrong; this is impressive stuff; but it needs to be an order of magnitude less 'screwing ar…

I used it for some troubleshooting in my job. Linux Sys Admin. I like how I can just ask it a question and explain the situation and it goes thru everything with me, like a Person.

Re: Artificial intelligence is losing hype

#354

Post by mrmetanoia »

mark_l_watson wrote: post #336

Yes and no. Hype over ‘API wrapper’ projects and startups will crash a bit, I think. On the other hand we are no where near approaching hard limits on LLMs. When LLMs start to be trained for smaller subject areas with massive hand curated examples for solving problems, then they will reach expert performance in those narrow tech areas. These specialized models will be combined in general purpose MoEs. Then new approa…

Yeah I think there's an empty marketing driven hype that will die off, but I think we're going to start to see it continue to integrate into peoples real life workflows and competition's going to heat up in delivering more consistently reliable results.

with regard to art AI, I think the debates are going to die off and the artists and people making stuff are going to just keep doing that, and some of them will use AI in ways that will challenge people in ways good art often does.

Re: Artificial intelligence is losing hype

#355

Post by mhuffman »

_acco wrote: post #294
layer8 wrote: post #249

Earlier quoted context omitted.

Does your work not depend on existing code bases, product architectures and nontrivial domain contexts the LLM knows nothing about? Every thread like this over the past year or so has had comments similar to yours, and it always remains quite vague, or when examples are given, it’s about self-contained tasks that require little contextual knowledge and are confined to widely publicly-documented technologies. What exa…

Context is the most challenging bit. FWIW, the codebases I'm working on are still small enough to where I rarely need to include more than 12 files into context. And I find as I make the context bigger beyond that, results degrade significantly. So I don't know how this would go in a much larger codebase. What floored him was simply how much of my programming I was doing with an LLM / how little I write line-by-line…

I know that you are getting some push-back because of your exuberance regarding your use of LLMs in development, but let me just say I respect that when someone told you to "put up or shut up" you did. Good on you!

Re: Artificial intelligence is losing hype

#356

Post by Rury »

JeremyNT wrote: post #253
greenthrow wrote: post #233

Earlier quoted context omitted.

Software engineer for multiple decades here. None of the AI assistants have made any major change to my job. They are useful tools when it's time to write code like many useful tools before them. But the hard work of being a senior+ software engineer comes before you start typing.

They help me most when using a framework, API, or language I'm not super familiar with. Beats stack overflow for that stuff. But it's weird to me seeing people talk about these changing their jobs so much. Maybe I'm holding it wrong but I'm almost always bottlenecked by "big picture" challenges and less on the act of actually typing the code.

That's because programming isn't just about syntax. The real difficulty lies in the problem solving. Sure AI can help with that, but the problem still largely rests on the programmer, as AI won't do anything unless prompted to.

Re: Artificial intelligence is losing hype

#357

Post by ikjasdlk2234 »

mark_l_watson wrote: post #336

Yes and no. Hype over ‘API wrapper’ projects and startups will crash a bit, I think. On the other hand we are no where near approaching hard limits on LLMs. When LLMs start to be trained for smaller subject areas with massive hand curated examples for solving problems, then they will reach expert performance in those narrow tech areas. These specialized models will be combined in general purpose MoEs. Then new approa…

I agree. We're seeing great results for very narrow use cases using smaller LLMs too. It's no different than classical ML and emerging AI over the past 10 years. If you don't have a well scoped use case, you're not going to succeed.

Now the one API wrapper projects that I love are my meeting transcription and summarization apps. You can tear those from my cold, dead hands.

Re: Artificial intelligence is losing hype

#358

Post by jholman »

slidehero wrote: post #315
bamboozled wrote: post #304

Earlier quoted context omitted.

I have teachers in my family, their lives have been basically ruined by people using ChatGPT-4 to cheat on their assignments. They spend their weekend trying to workout if someone has "actually written" this or not. So sorry, we're back to spam generator. Even if it's "good spam".

>their lives have been basically ruined a bit dramatic. there has to be an adjustment of teaching/assessing, but nothing that would "ruin" anyone's life. >So sorry, we're back to spam generator. Even if it's "good spam". is it spam if it's useful and solves a problem? I don't agree it fits the definition any more. Teachers are under immense pressure, GPT allows a teacher to generate extension questions for gifted stu…

> a bit dramatic. there has to be an adjustment of teaching/assessing, but nothing that would "ruin" anyone's life.

If you don't have the power to just change your mind about what the entire curriculum and/or assessment context is, it can be a workload increase of dozens of hours per week or more. If you do have the power, and do want to change your entire curriculum, it's hundreds of hours one-time. "Lives basically ruined" is an exaggeration, but you're preposterously understating the negative impact.

> is it spam if it's useful and solves a problem?

Whether or not it's useful has nothing to do with whether or not it's spam. I'm not claiming that your product is spam -- I'll get back to that -- but your reply to the spam accusation is completely wrong.

As for your hypothesis, I've had interactions where it did a good job of generating alternative activities/exercises, and interactions where it strenuously and lengthily kept suggesting absolute garbage. There's already garbage on the internet, we don't need LLMs to generate more. But yes, I've had situations where I got a good suggestion or two or three, in a list of ten or twenty, and although that's kind of blech, it's still better than not having the good suggestions.

Re: Artificial intelligence is losing hype

#359

Post by Calavar »

_acco wrote: post #207

AI (specifically Claude Sonnet via Cursor) has completely transformed my workflow. It's changed my job description as a programmer. (And I've been doing this for 13y – no greenhorn!) This wasn't the case with GPT-4/o. This capability is very new. When I spoke to a colleague at Microsoft about these changes, they were floored. Microsoft has made themselves synonymous with AI, yet their company is barely even leveragin…

I feel like I'm living in a different universe sometimes. The consensus on HN seems to be that you can be pretty productive with LLMs as coding assistants, but every time I try I find it borderline impossible to get functional code even for pretty straightforward prompts.

I decided to fire up GPT-4o again today to see if maybe things have gotten better over the past few months.

I asked GPT to write code to render a triangle using Vulkan (a 3D graphics API). There are about 1000 tutorials on this that are almost certainly in GPT-4's training data. I gave GPT two small twists so it's not a simple case of copy/paste: I asked it 1) to apply a texture to the triangle and 2) to keep all the code in a single function. (Most tutorials break the code up into about a dozen functions, but each of these functions are called only once, so it should be trivial to inline them.)

Within the first ten lines, the code is already completely nonfunctional:

GPT-4o declares a pointer (VkPhysicalDevice) that is uninitialized. It queries the number of graphics devices on the host machine. A human being would allocate a buffer with that number of elements and store the reference in the pointer. GPT-4o just ignores the result. Completely ignores it. So the function call was just for fun, I guess? It then tries to copy an entire array of VkPhysicalDevice_T objects into this uninitialized pointer. So that's a guaranteed memory access violation right off the bat.

Re: Artificial intelligence is losing hype

#360

Post by codexon »

MaybiusStrip wrote: post #329
codexon wrote: post #322

Earlier quoted context omitted.

I'm not going to pass any judgement on what is reasonable or unreasonable for LLMs, just pointing out that it makes serious mistakes. Many people here argue that LLMs are able to reason, you can check my post history 3 months ago to see an example of this. And if you have to double check everything, how much time are you really saving? And is this thing really on track to replace programmers any time soon? Absolutely…

I'm confused, you said earlier that you use it every day.

Where did I say I didn't?
Post reply on HN