Live data from Hacker News

My AI skeptic friends are all nuts

fly.io

501–510 of 1001 posts

Re: My AI skeptic friends are all nuts

#501

I find the Konwinski Prize to be very interesting in this context. 1 million dollars to whoever's open source LLM solves >90% of a set of novel Github issues. https://www.kaggle.com/competitions/konwinski-prize/ Currently, the #1 spot sits at a score of 0.09, not 0.9. A far cry from being useful. I know that open source models are not as good as closed source, but still, we're a long way from LLMs being good for code…

> Currently, the #1 spot sits at a score of 0.09, not 0.9. A far cry from being useful.

The best intellisense and code completion tools would solve 0.00. Those were the only tools we were using just a couple of years ago. 0.09 is a tremendous jump and the improvements will accelerate!

Re: My AI skeptic friends are all nuts

#502
post #6

One thing that I find truly amazing is just the simple fact that you can now be fuzzy with the input you give a computer, and get something meaningful in return. Like, as someone who grew up learning to code in the 90s it always seemed like science fiction that we'd get to a point where you could give a computer some vague human level instructions and get it more or less do what you want.

There's the old quote from Babbage:

> On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.

This has been an obviously absurd question for two centuries now. Turns out the people asking that question were just visionaries ahead of their time.

It is kind of impressive how I'll ask for some code in the dumbest, vaguest, sometimes even wrong way, but so long as I have the proper context built up, I can get something pretty close to what I actually wanted. Though I still have problems where I can ask as precisely as possible and get things not even close to what I'm looking for.

Re: My AI skeptic friends are all nuts

#504

This isn't about the post itself, but can you please put the author's name at the top? It's written in first person and we have no idea who's talking until the end of the post.

To anyone else who was going to protest that the name _is_ at the top, apparently it depends on one's browser window width and zoom level. I was a bit surprised to find that increasing the zoom level even affects the order when reading with a screen reader.

Re: My AI skeptic friends are all nuts

#506
I love LLMs, and I really like programming with Cursor, but I never managed to get the "agents with tons of stuff in their context" mode to work for me. I use Cursor like a glorified code completer, 4-5 lines at a time, because otherwise the LLM just makes too many mistakes that compound.

If you let it run in the "write my code for me" mode, and ask it to fix some mistake it made, it will always add more code, never remove any. In my experience, in the end the code just ends up so brittle that the LLM will soon get stuck at a point that it never manages to overcome some mistake, no matter how many times it tries.

Has anyone managed to solve this?

Re: My AI skeptic friends are all nuts

#507
post #3

Machine translation and speech recognition. The state of the art for these is a multi-modal language model. I'm hearing impaired veering on deaf, and I use this technology all day every day. I wanted to watch an old TV series from the 1980s. There are no subtitles available. So I fed the show into a language model (Whisper) and now I have passable subtitles that allow me to watch the show. Am I the only one who remem…

would be interesting if court transcriptions can be handled by these models.

Re: My AI skeptic friends are all nuts

#508
I think the key premise here is that one can effectively and efficiently audit code that the LLM is producing.

I doubt that. First, human attention and speed is very limited. Second, when I see something, I am already predisposed to assume that it is right (or at the very least, my subsequent inquiries are extremely narrow and anchored around the solution I have seen presented to me.)

Re: My AI skeptic friends are all nuts

#509
post #428

Hundreds of comments. Some say LLMs are the future. Others say they don't work today and they won't work tomorrow. Videogame speed running has this problem solved. Livestream your 10x engineer LLM usage, a git commit annotated with it's prompt per change. Then everyone will see the result. This doesn't seem like an area of debate. No complicated diagrams required. Just run the experiment and show the result.

I'd honestly love to see this.

People always say "you just need to learn to prompt better" without providing any context as to what "better" looks like. (And, presumes that my prompt isn't good enough, which maybe it is maybe it isn't.)

The easy way out of that is "well every scenario is different" - great, show me a bunch of scenarios on a speed run video across many problems, so I can learn by watching.

Re: My AI skeptic friends are all nuts

#510
The premise of “lol, I’m just a dumb developer and just wait until the smart developers catch on” falls extremely flat coming from Thomas.

It kinda invalidates the entire premise because it builds on this false foundation of “look how useful it is to a mere plebeian” and the skills actually required to sift through LLM garbage are what puts you in the top 5% of programmers.

Most senior developers cannot conduct useful code review and spot errors. In every company I’ve been at, there were only 1 in 20 people that would give the useful in-depth reviews required to pick up LLM bullshit.

This style of engagement bait is brutal and it severely overshadows other nuggets of wisdom of how LLMs can be useful.

Post reply on HN