Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I genuinely don't understand why some people are still bullish about LLMs

twitter.com

Post reply on HN
Print view View original

101–110 of 1001 posts

Re: I genuinely don't understand why some people are still bullish about LLMs

#101

Post by saaaaaam »

pesus wrote: post #92
saaaaaam wrote: post #26

I’ve used Claude today to: Write code to pull down a significant amount of public data using an open API. (That took about 30 seconds - I just gave it the swagger file and said “here’s what I want”) Get the data (an hour or so), clean the data (barely any time, gave it some samples, it wrote the code), used the cleaned data to query another API, combined the data sources, pulled down a bunch of PDFs relating to the d…

This sounds like a lot of actions without any verification that the LLM didn't misinterpret things or just make something up.

Well the API calls worked perfectly. The LLM didn’t misinterpret that.

The data extraction via tesseract worked too.

The whisper transcript was pretty good. Not perfect, but when you do this daily you are easily able to work around things.

The summaries of the calls were very useful. I could easily verify those because I was on the calls.

The interview - again, transcript is great. The bulleted narrative was guided - again - by me having been on the call. I certify he quotes against the transcript, and audio if I’ve got any doubts.

Scrapers - again, they worked fine. The LLM didn’t misinterpret anything.

Podcasts - as before. Easy.

Article to voice - what’s to misinterpret?

Your criticism sounds like a lot of waffle with no understanding of how to use these tools.

Re: I genuinely don't understand why some people are still bullish about LLMs

#102

Post by asadotzler »

jongjong wrote: post #7

People who don't work in tech have no idea how hard it is to do certain things at scale. Skilled tech people are severely underappreciated. From a sub-tweet: >> no LLM should ever output a url that gives a 404 error. How hard can it be? As a developer, I'm just imagining a server having to call up all the URLs to check that they still exist (and the extra costs/latency incurred there)... And if any URLs are missing,…

It's all done quite easily when that's the priority but it's not because the priority is hype to keep the cycle going to mint a few more ultra-wealthy assholes. Nothing about fixing 404s would require resources unavailable to these mega-corporations and your and others carrying their water is part of why they don't.

Re: I genuinely don't understand why some people are still bullish about LLMs

#103

Post by prennert »

I have been using Claude this week the first time for a _slightly_ bigger SwiftUI project than just a few lines of bash or SQL I used it before. I have never used swift before but I am amazed how much Claude could do. It feels to me as we are at the point where anyone can now generate small tools with low effort for themselves. Maybe not production ready, but good enough to use yourself. It feels like it should be good enough to empower the average user to break out of having to rely on pre-made apps to do small things. Kind of like bash for the average Joe.

What worked:

- generated a mostly working PoC with minimal input and hallucinated UI layout, Color scheme, etc. this is amazing because it did not bombard me with detailed questions. It just carried on to provide me with a baseline that I could then finetune

- it corrected build issues by me simply copy pasting the errors from Xcode - got APIs working - added debug code when it could not fix an issue after a few rounds

- resolved an API issue after I pointed it to a typescript SDK to the API (I literally gave a link to the file and told it, try to use this to work out where the problem is) - it produces code very fast

What is not working great yet:

- it started off with one large file and crashed soon after because it hit a timeout when regenerating the file. I needed to ask it to split the file into a typical project order

- some logic I asked it to implement explicitly got changed at some point during an unrelated task. To prevent this in future I asked it mark this code part as important and that it should only be changed at explicit request. I don’t know yet how long this code will stay protected for

- by the time enough context got build up usage warnings pop up in Claude

- only so many files are supported atm

So my takeaway is that it is very good at translating, I.e. API docs into code, errors into fixes. There is also a fine line between providing enough context and running out of tokens.

I am planning to continue my project to see how far I can push it. As I am getting close to the limit of the token size now, I am thinking of structuring my app in a Claude friendly way:

- clear internal APIs. Kind of like header files so that I can tell Claude what functions it can use without allowing it to change them or needing to tokenize the full source code

- adversarial testing. I don’t have tests yet, but I am thinking of asking one dedicated instance of Claude to generate tests. I will use other Claude instances for coding and provide them with failing test outputs like I do now with build errors. I hope it will fix itself similarly.

Re: I genuinely don't understand why some people are still bullish about LLMs

#104

Post by prawn »

doctoboggan wrote: post #28

Why do people who don't like using LLMs keep insisting they are useless for the rest of us? If you don't like to use them, then simply don't use them. I use them almost daily in my job and get tremendous use out of them. I guess you could accuse me of lying, but what do I stand to gain from that? I've also seem people claim that only people who don't know how to code or people doing super simple done a million times…

Even outside of work. Two personal examples, out of dozens this week:

1. Asked ChatGPT for a table showing monthly daily max temp, rainfall in mm and numbers of rain days, for Vietnam, Cambodia and Thailand. And colour coded based on the temperatures. Then suggest two times of year, and a route direction, to hit the best conditions on a multi-week trip.

It took a couple of seconds, and it helpfully split Vietnam at Hanoi and HCM given their weather differences.

2. I'm trying to work out how I will build a chicken orchard - post material, spacing, mesh, etc. I asked ChatGPT for a comparison table of steel posts versus timber, and then to cost it out with varying scales of post spacing. Plus pros and cons of each, and likely effort to build. Again, it took a few seconds, including browsing local stores for indicative pricing.

On top of that, I've been even more impressed by a first week testing Cursor.

Re: I genuinely don't understand why some people are still bullish about LLMs

#105

Post by mcv »

islewis wrote: post #40

> I genuinely don't understand why some people are still bullish about LLMs. I don't believe OP's thesis is properly backed by the rest of his tweet, which seems to boil down to "LLM's can't properly cite links". If LLM's performing poorly on an arbitrary small-scoped test case makes you bearish on the whole field, I don't think that falls on the LLM's.

Her point is not just "LLMs can't cite links", but "LLMs make shit up". And that is absolutely a problem.

Re: I genuinely don't understand why some people are still bullish about LLMs

#106

Post by bluebarbet »

Perhaps attitudes to this new phenomenon are correlated with propensity to skepticism in general.

I will cite myself as Exhibit A. I am the sort of person who takes almost nothing at face value. To me, physiotherapy, and oenology, and musicology, and bed marketing, and mineral-water benefits, and very many other such things, are all obviously pseudoscience, worthy of no more attention than horoscopes. If I saw a ghost I would assume it was a hallucination caused by something I ate.

So it seems like no coincidence that I reflexively ignore the AI babble at the top of search results. After all, an LLM is a language-rehashing machine which (as we all know by now) does not understand facts. That's terribly relevant.

I remember reading, a couple of years back, about some Very Serious Person (i.e. a credible voice, I believe some kind of scientist) who, after a three-hour conversation with ChatGPT, had become convinced that the thing was conscious. Rarely have I rolled my eyes so hard. It occurred to me then that skepticism must be (even) less common a mindset than I assumed.

Re: I genuinely don't understand why some people are still bullish about LLMs

#107

Post by PlunderBunny »

dale_glass wrote: post #63
turtletontine wrote: post #30

Earlier quoted context omitted.

I literally had a developer of an open source package I’m working with tell me “yeah that’s a known problem, I gave up on trying to fix it. You should just ask ChatGPT to fix it, I bet it will immediately know the answer.” Annoying response of course. But I’d never used an LLM to debug before, so I figured I’d give it a try. First: it regurgitated a bunch of documentation and basic debugging tips, which might have ac…

On the other hand, when it works it's darn near magic. I spent like a week trying to figure out why a livecd image I was working on wasn't initializing devices correctly. Read the docs, read source code, tried strace, looked at the logs, found forums of people with the same problem but no solution, you know the drill. In desperation I asked ChatGPT. ChatGPT said "Use udevadm trigger". I did. Things started working. F…

> For some problems it's just very hard to express them in a googleable form

I had an issue where my Mac would report that my tethered iPhone's batteries were running low when the battery was in fact fine. I had tried googling an answer, and found many similar-but-not-quite-the-same questions and answers. None of the suggestions fixed the issue.

I then asked the 'MacOS Guru' model for chatGPT my question, and one of the suggestions worked. I feel like I learned something about chatGPT vs Google from this - the ability of an LLM to match my 'plain English question without a precise match for the technical terms' is obviously superior to a search engine. I think google etc try synonyms for words in the query, but to me it's clear this isn't enough.

Re: I genuinely don't understand why some people are still bullish about LLMs

#108

Post by taberiand »

Sohcahtoa82 wrote: post #31
MostlyStable wrote: post #4

My experience (almost exclusively Claude), has just been so different that I don't know what to say. Some of the examples are the kinds of things I explicitly wouldn't expect LLMs to be particularly good at so I wouldn't use them for, and others, she says that it just doesn't work for her, and that experience is just so different than mine that I don't know how to respond. I think that there are two kinds of people w…

> LLMs are almost perfect for this. It's generally faster than me looking up syntax/documentation, when it's wrong it's easy to tell and correct. Exactly this. I once had a function that would generate several .csv reports. I wanted these reports to then be uploaded to s3://my_bucket/reports/{timestamp}/ .csv I asked ChatGPT "Write a function that moves all .csv files in the current directory to and old_reports direc…

I have been saying this about llms for a while - if you know what you want, how to ask for it, and what the correct output will look like, LLMs are fantastic (at least Claude Sonnet is). And I mean that seriously, they are a highly effective tool for productive development for senior developers.

I use it to produce whole classes, large sql queries, terraform scripts, etc etc. I then look over that output, iterate on it, adjust it to my needs. It's never exactly right at first, but that's fine - neither is code I write from scratch. It's still a massive time saver.

Re: I genuinely don't understand why some people are still bullish about LLMs

#109

Post by gilbetron »

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me.

And people just sit around, unimpressed, and complain that ... what ... it isn't a perfect superintelligence that understands everything perfectly? This is the most amazing technology I've experienced as a 50+ year old nerd that has been sitting deep in tech for basically my whole life. This is the stuff of science fiction, and while there totally are limitations, the speed at which it is progressing is insane. And people are like, "Wah, it can't write code like a Senior engineer with 20 years of experience!"

Crazy.

Re: I genuinely don't understand why some people are still bullish about LLMs

#110

Post by meowface »

gilbetron wrote: post #109

I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…

A lot of software developers have an initial bad experience and assume it's terrible and give up on it.

I feel bad for people who haven't yet experienced how useful these models are for programming.

Some also just prefer manually entering everything. Those people I will never understand.

Post reply on HN