Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory and new controls for ChatGPT

openai.com

Post reply on HN
Print view View original

91–100 of 270 posts

Re: Memory and new controls for ChatGPT

#91

Post by anthonypasq »

nafizh wrote: post #86
OJFord wrote: post #62

Earlier quoted context omitted.

I haven't really tried to use it for coding, other than once (recently, so not before some decline) indirectly, which I was pretty impressed with: I asked about analyst expectations for the Bank of England base rate, then asked it to compare a fixed mortgage with a 'tracker' (base rate + x; always x points over the base rate). It spat out the repayment figures and totals over the two years, with a bit of waffle, and…

To be honest, when I say it has significantly worsened, I am comparing to the time when GPT-4 just came out. It really felt like we were on the verge of 'AGI'. In 3 hours, I coded up a complex piece of web app with chatgpt which completely remembered what we have been doing the whole time. So, it's sad that they have decided against the public having access to such strong models (and I do think it's intentional, not…

i mean i feel like its fairly plausible that the smarter model costs more, and access to GPT-4 is honestly quite cheap all thing considered. Maybe in the future theyll have more price tiers.

Re: Memory and new controls for ChatGPT

#92

Post by Kranar »

nafizh wrote: post #38

My use of ChatGPT has just organically gone down 90%. It's unable to do any sort of task of non-trivial complexity e.g. complex coding tasks, writing complex prose that conforms precisely to what's been asked etc. Also I hate the fact that it has to answer everything in bullet points, even when it's not needed, clearly rlhf-ed. At this point, my question types have become what you would ask a tool like perplexity.

Sure, but consider not using it for complex tasks. My productivity has skyrocketed with ChatGPT precisely because I don't use it for complex tasks, I use it to automate all of the trivial boilerplate stuff.

ChatGPT writes excellent API documentation and can also document snippets of code to explain what they do, it does 80% of the work for unit tests, it can fill in simple methods like getters/setters, initialize constructors, I've even had it write a script to perform some substantial code refactoring.

Use ChatGPT for grunt work and focus on the more advanced stuff yourself.

Re: Memory and new controls for ChatGPT

#93

Post by makestuff »

th0ma5 wrote: post #88
anotherpaulg wrote: post #51

This is a bit off topic to the actual article, but I see a lot of top ranking comments complaining that ChatGPT has become lazy at coding. I wanted to make two observations: 1. Yes, GPT-4 Turbo is quantitatively getting lazier at coding. I benchmarked the last 2 updates to GPT-4 Turbo, and it got lazier each time. 2. For coding, asking GPT-4 Turbo to emit code changes as unified diffs causes a 3X reduction in lazy co…

How is laziness programmatically defined or used as a benchmark

Personally I have seen it saying stuff like:

public someComplexLogic() { // Complex logic goes here }

or another example when the code is long (ex: asking it to create a vue component) is that it will just add a comment saying the rest of the code goes here.

So you could test for it by asking it to create long/complex code and then running the output against unit tests that you created.

Re: Memory and new controls for ChatGPT

#94

Post by joshspankit »

Is there anything revolutionary about this “memory” feature?

Looks like it’s just summarizing facts gathered during chats and adding those to the prompt they feed to the AI. I mean that works (been doing it myself) but what’s the news here?

Re: Memory and new controls for ChatGPT

#95

Post by asaddhamani »

Me1000 wrote: post #89
omalled wrote: post #87

Earlier quoted context omitted.

Can you say in one or two sentences what you mean by “lazy at coding” in this context?

It has a tendency to do: "// ... the rest of your code goes here" in it's responses, rather than writing it all out.

It's incredibly lazy. I've tried to coax it into returning the full code and it will claim to follow the instructions while regurgitating the same output you complained about. GPT-4 was great, GPT-4 Turbo first version was pretty terrible bordering on unusable, then they came out with the Turbo second version, which almost feels worse to me, though I haven't compared, but if someone comes claiming they fixed an issue, but you still see it, it will bias you to see it more.

Claude is doing much better in this area, local/open LLMs are getting quite good, it feels like OpenAI is not heading in a good direction here, and I hope they course correct.

Re: Memory and new controls for ChatGPT

#96

Post by asaddhamani »

minimaxir wrote: post #45
bluish29 wrote: post #28

It is already ignoring your prompt and custom instructions. For example, If I explicity ask it to provide a code instead of an overview it will respond by apologizing and then provide the same overview answer with minimal if no code. Will memory provide a solution to that or will be a different thing to ignore?

Did you try promising it a $500 tip for behaving correctly? (not a shitpost: I'm working on a more academic analysis of this phenomenon)

I have tried this after seeing it recommended in various forums, it doesn't work. It says things like:

"I appreciate your sentiment, but as an AI developed by OpenAI, I don't have the capability to accept payments or incentives."

Re: Memory and new controls for ChatGPT

#98

Post by vb234 »

schmichael wrote: post #81

> As a kindergarten teacher with 25 students, you prefer 50-minute lessons with follow-up activities. ChatGPT remembers this when helping you create lesson plans. Somebody needs to inform OpenAI how Kindergarten works... classes are normally smaller than that, and I don't think any kindergarten teacher would ever try to pull off a "50-minute lesson." Maybe ai wrote this list of examples. Seems like a hallucination wh…

Indeed. Thanks to snow day here in NYC, my first grader has remote learning and all academic activity (reading, writing and math) was restricted to 20 minutes in her learning plan.

Re: Memory and new controls for ChatGPT

#100

Post by rvnx »

makestuff wrote: post #93
th0ma5 wrote: post #88

Earlier quoted context omitted.

How is laziness programmatically defined or used as a benchmark

Personally I have seen it saying stuff like: public someComplexLogic() { // Complex logic goes here } or another example when the code is long (ex: asking it to create a vue component) is that it will just add a comment saying the rest of the code goes here. So you could test for it by asking it to create long/complex code and then running the output against unit tests that you created.

Yeah this is a typical issue:

- Can you do XXX (something complex) ?

- Yes of course, to do XXX, you need to implement XXX, and then you are good, here is how you can do:

int main(int argc, char **argv) {

  /* add your implementation here */

}
Post reply on HN