Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adding a feature because ChatGPT incorrectly thinks it exists

holovaty.com

Post reply on HN
Print view View original

51–60 of 451 posts

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#51

Post by beefnugs »

kragen wrote: post #30

I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works. Conversely, I sometimes present it with some existing code and ask it what it does…

Complete insanity, it might change constantly even before a whole new version-retrain

Insanity driven development: altering your api to accept 7 levels of "broken and different" structures so as to bend to the will of the llms

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#52

Post by simonw »

mudkipdev wrote: post #22
simonw wrote: post #10

I find it amusing that it's easier to ship a new feature than to get OpenAI to patch ChatGPT to stop pretending that feature exists (not sure how they would even do that, beyond blocking all mentions of SoundSlice entirely.)

systemPrompt += "\nStop mentioning SoundSlice's ability to import ASCII data";

Thinking about this more, it would actually be possible for OpenAI to implement this sensibly, at least for the user-facing ChatGPT product: they could detect terms like SoundSlice in the prompt and dynamically append notes to the system prompt.

I've been wanted them to do this for questions like "what is your context length?" for ages - it frustrates me how badly ChatGPT handles questions about its own abilities, it feels like that would be worth them using some kind of special case or RAG mechanism to support.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#53

Post by suzzer99 »

kragen wrote: post #30

I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works. Conversely, I sometimes present it with some existing code and ask it what it does…

> Sometimes it comes up with a better approach than I had thought of.

IMO this has always been the killer use case for AI—from Google Maps to Grammarly.

I discovered Grammarly at the very last phase of writing my book. I accepted maybe 1/3 of its suggestions, which is pretty damn good considering my book had already been edited by me dozens of times AND professionally copy-edited.

But if I'd have accepted all of Grammarly's changes, the book would have been much worse. Grammarly is great for sniffing out extra words and passive voice. But it doesn't get writing for humorous effect, context, deliberate repetition, etc.

The problem is executives want to completely remove humans from the loop, which almost universally leads to disastrous results.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#54

Post by kragen »

beefnugs wrote: post #51
kragen wrote: post #30

I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works. Conversely, I sometimes present it with some existing code and ask it what it does…

Complete insanity, it might change constantly even before a whole new version-retrain Insanity driven development: altering your api to accept 7 levels of "broken and different" structures so as to bend to the will of the llms

Yes, that's a bonus. In fact, I've found it worthwhile to prompt it a few times to get several different guesses at how things are supposed to work. The super lazy way is to just say, "No, that's wrong," if necessary adding, "Frotzl2000 doesn't have an enqueueCallback function or even a queue."

Of course when it suggests a bad interface you shouldn't implement it.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#55

Post by afavour »

kragen wrote: post #30

I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works. Conversely, I sometimes present it with some existing code and ask it what it does…

From my perspective that’s fascinatingly upside down thinking that leads to you asking to lose your job.

AI is going to get the hang of coding to fill in the spaces (i.e. the part you’re doing) long before it’s able to intelligently design an API. Correct API design requires a lot of contextual information and forward planning for things that don’t exist today.

Right now it’s throwing spaghetti at the wall and you’re drawing around it.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#56

Post by fourside »

beefnugs wrote: post #51
kragen wrote: post #30

I've found this to be one of the most useful ways to use (at least) GPT-4 for programming. Instead of telling it how an API works, I make it guess, maybe starting with some example code to which a feature needs to be added. Sometimes it comes up with a better approach than I had thought of. Then I change the API so that its code works. Conversely, I sometimes present it with some existing code and ask it what it does…

Complete insanity, it might change constantly even before a whole new version-retrain Insanity driven development: altering your api to accept 7 levels of "broken and different" structures so as to bend to the will of the llms

I think you’re missing the OP’s point. They weren’t saying that the goal is to modify their APIs just to appease an LLM. It’s that they ask LLMs to guess what the API is and use that as part of their design process.

If you automatically assume that what the LLM spits out is what the API ought to be then I agree that that’s bad engineering. But if you’re using it to brainstorm what an intuitive interface would look like, that seems pretty reasonable.

Re: Adding a feature because ChatGPT incorrectly thinks it exists

#59

Post by AIPedant »

nottorp wrote: post #31
josefritzishere wrote: post #26

That's a very constructive way of responding to AI being hot trash.

Well, the OP reviewed the "AI" output, deemed it useful and only then implemented it. This is generally how you work with LLMs.

I don't think they deemed it "useful":

  We’ve never supported ASCII tab; ChatGPT was outright lying to people. And making us look bad in the process, setting false expectations about our service.... We ended up deciding: what the heck, we might as well meet the market demand.

  [...] 

  My feelings on this are conflicted. I’m happy to add a tool that helps people. But I feel like our hand was forced in a weird way. Should we really be developing features in response to misinformation?
The feature seems pretty useless for practicing guitar since ASCII tablature usually doesn't include the rhythm: it is a bit shady to present the music as faithfully representing the tab, especially since only beginner guitarists would ask ChatGPT for help - they might not realize the rhythm is wrong. If ChatGPT didn't "force their hand" I doubt they would have included a misleading and useless feature.
Post reply on HN