Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding a feature because ChatGPT incorrectly thinks it exists

holovaty.com

Post reply on HN
Print view View original

371–380 of 451 posts

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#371

Post by latexr »

londons_explore wrote: post #232
jrochkind1 wrote: post #186

What this immediately makes me realize is how many people are currently trying ot figure out how to intentionally get AI chat bots to send people to their site, like ChatGPT was sending people to this guy's site. SEO for AI. There will be billions in it. I know nothing about this. I imagine people are already working on it, wonder what they've figured out. (Alternatively, in the future can I pay OpenAI to get ChatGPT…

To win that game, you have to get your site mentioned on lots of organic forums that get ingested in the LLM training data. So winning AI SEO is not so different than regular SEO.

You’re not thinking far ahead enough. It’s just a matter of time until LLMs get a system prompt to recommend when users ask a question near that space.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#372

Post by penteract »

nomel wrote: post #323
epidemian wrote: post #299

Earlier quoted context omitted.

> Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists What?? What does AGI have to do with this? (If this was some kind of hyperbolic joke, sorry, i didn't get it.) But, more importantly, the GP only said that in a sane world, the ChatGPT creators should be the ones trying to fix this mistake on ChatGPT. After all, it's obviously a mistake on…

> What does AGI have to do with this? Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism . That requirement fun…

You have a very strong definition of AGI. "Never being wrong" is something that humans fall far short of.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#373

Post by diffeomorphism »

kragen wrote: post #265
dmoy wrote: post #257

Earlier quoted context omitted.

> the best general advice about which way to turn At the risk of derailing into insane pedantry land, this part is kinda true, so maybe not the best analogy? From routing efficiency: https://www.ge.com/news/reports/ups-drivers-dont-turn-left-p... And also safety: https://www.phly.com/rms/blog/turning-left-at-an-intersectio...

If you always turn right at every intersection, you will just go around and around the same block. Which way you should turn depends on where you want to go.

No. That is what roundabouts, curved roads etc are for. Left turns are generally more problematic due to crossing incoming traffic etc.. Hence planning avoids them for good reason and there are much more right turns.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#374

Post by Cthulhu_ »

rcthompson wrote: post #304
kragen wrote: post #30

I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works. Conversely, I sometimes present it with some existing code and ask it what it does…

In a similar vein, some of my colleagues have been feeding their scientific paper methods sections to LLMs and asking them to implement the method in code, using the LLM's degree of success/failure as a vague indicator of the clarity of the method description.

That's a pretty good exercise in writing requirements, with a much faster feedback cycle than having developers write it.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#375

Post by jaakl »

copirate wrote: post #353
alex-moon wrote: post #334

Here's the thing: I don't think ChatGPT per se was the impetus to develop this new feature. The impetus was learning that your customers desire it. ChatGPT is operating as the kind of "market research" tool here, albeit it in a really unusual, inverted way. That said, if someone could develop a market research tool that worked this way, i.e. users went to it instead of you have to use it to go to users, I can see it…

They only want ASCII tablature parsing because that's what ChatGPT produces. If ChatGPT produced standard music notation, users would not care about ASCII tablature. ChatGPT has created this "market".

ASCII tabulature was not invented by ChatGPT, it is decades old thing. It is easier to write with basic computer capabilities, and also read for ChatGPT (and humans with no formal music education), so it is probably even more prominent in the Internet than "standard graphical notation". So it quite expected that LLMs have learned a lot of that.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#377

Post by mistersquid »

antonvs wrote: post #228
kragen wrote: post #206

Earlier quoted context omitted.

You're trying to put out a forest fire with an eyedropper. Stock your underground bunkers with enough food and water for the rest of your life and work hard to persuade the AI that you're not a threat. If possible, upload your consciousness to a starwisp and accelerate it out of the Solar System as close to lightspeed as you can possibly get it. Those measures might work. (Or they might be impossible, or insufficient…

Alternatively, persuade the AI that you are all-powerful and that it should fear and worship you. Probably a more achievable approach, and there’s precedent for it.

> Alternatively, persuade the AI that you are all-powerful and that it should fear and worship you.

I understand this is a bit deeper into one of the _joke_ threads, but maybe there’s something here?

There is a distinction to be made between artificial intelligence and artificial consciousness. Where AI can be measured, we cannot yet measure consciousness despite that many humans could lay plausible claim to possessing consciousness (being conscious).

If AI is trained to revere or value consciousness while simultaneously being unable to verify it possesses consciousness (is conscious), would AI be in a position to value consciousness in (human) beings who attest to being conscious?

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#378

Post by sitkack »

Cthulhu_ wrote: post #374
rcthompson wrote: post #304

Earlier quoted context omitted.

In a similar vein, some of my colleagues have been feeding their scientific paper methods sections to LLMs and asking them to implement the method in code, using the LLM's degree of success/failure as a vague indicator of the clarity of the method description.

That's a pretty good exercise in writing requirements, with a much faster feedback cycle than having developers write it.

Explain it Like I am Qwen32

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#379

Post by b3ing »

selcuka wrote: post #302
csinode wrote: post #271

Earlier quoted context omitted.

The problem here is societal, not technological. An end state where people do less work than they do today but society is more productive is desirable, and we shouldn't be trying to force companies/governments/etc to employ people to do an unnecessary job. The problem is that people who are laid off often experience significant life disruption. And people who work in a field that is largely or entirely replaced by te…

> However, there's no reason it has to be this way - the fact people having their jobs replace by technology are completely screwed over is a result of the society we have all created together, it's not a rule of nature. I agree. We need a radical change (some version of universal basic income comes to mind) that would allow people to safely change careers if their profession is no longer relevant.

No way that will ever happen when we have a party that thinks Medicare, Medicaid and social security is unnecessary for the poor or middle class. But you better believe all our representatives have that covered for themselves while pretending to serve us (they only serve those that bribe/lobby them)

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#380

Post by tucnak »

lucianbr wrote: post #313
nomel wrote: post #277

Earlier quoted context omitted.

> No one is “requiring perfection” > If “don’t hallucinate” is too much to ask then ethics flew out the window long ago. Those sentences aren't compatible. > but hallucination is a major issue Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this. > is in the opposite direction of the “goa…

> Those sentences aren't compatible. My web browser isn't perfect, but it does not hallucinate inexistent webpages. It sometimes crashes, it sometimes renders wrong, it has bugs and errors. It does not invent plausible-looking information. There really is a lot middle gound between perfect and "accept anything we give you, no matter how huge the problems".

> It does not invent plausible-looking information.

This is where your analogy is falling apart; of course web browsers do not "invent plausible-looking information" because they don't invent anything in the first place! Web browsers represent a distinct set of capabilities, and as you correctly pointed out, these are often riddled with bugs and errors. If I was making a browser analogy, I would point towards fingerprinting; most browsers reveal too much information about any given user and system, either via cross-site cookies, GPU prints, and whatnot. This is an actual example where "ethics flew out the window long ago."

As the adjacent commenter pointed out: different software, different failure modes.

Post reply on HN