GPT-4.5 or GPT-5 being tested on LMSYS? • Hacker News Board

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#11

Post by xennygrimmato » Mon, Apr 29, 2024, 5:22 PM UTC

What's interesting it seems to also edit text in between for me, not just add text at the end.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#12

Post by steve_adams_86 » Mon, Apr 29, 2024, 5:23 PM UTC

atemerev wrote: post #6

Indeed, all my test prompts are giving much better results than gpt4-turbo and Claude Opus. And yet, the OpenAI style is clearly recognizable.

I agree. It responds a lot like GPT-4 in style, but with much better answers.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#13

Post by drno123 » Mon, Apr 29, 2024, 5:23 PM UTC

Still gives incorrect code to the following prompt - the description is correct but not the code. I have yet to find one LLM that gives the correct code. This is the prompt:

“Write C code to calculate the sum of matrix elements below the secondary diagonal.“

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#14

Post by gigel82 » Mon, Apr 29, 2024, 5:24 PM UTC

Holy shit, the results are amazing; I didn't think they could still make exponential leaps in quality...

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#15

Post by kurtextrem » Mon, Apr 29, 2024, 5:26 PM UTC

It tells me that its knowledge cutoff is Nov 2023. So pretty recent.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#16

Post by swOOnk » Mon, Apr 29, 2024, 5:26 PM UTC

Prompt to gpt2-chatbot: who are you? and which version?

Answer: Hello! I'm ChatGPT, a language model developed by OpenAI based on the GPT-4 architecture. I'm here to help answer your questions, assist with a wide range of topics, or just chat about whatever's on your mind! How can I assist you today?

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#17

Post by jessenaser » Mon, Apr 29, 2024, 5:28 PM UTC

OpenAI team, if you are reading this I am impressed and thank you for your work. Can't wait to see the official release of GPT-next.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#18

Post by jonchurch_ » Mon, Apr 29, 2024, 5:28 PM UTC

kurtextrem wrote: post #15

It tells me that its knowledge cutoff is Nov 2023. So pretty recent.

gpt-4 latest is December 2023, just asked it on chat.openai.com

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#19

Post by msoad » Mon, Apr 29, 2024, 5:29 PM UTC

Prompt: code up an analog clock in html/js/css. make sure the clock is ticking exactly on the second change. second hand red. other hands black. all 12 hours marked with numbers.

ChatGPT-4 Results: https://jsbin.com/giyurulajo/edit?html,css,js,output

GPT2-Chatbot Results: https://jsbin.com/dacenalala/2/edit?html,css,js,output

Claude3 Opus Results: https://jsbin.com/yifarinobo/edit?html,css,js,output

None is correct. Styling is off in all, each in a different way. And all made the mistake of not ticking when second actually changes

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#20

Post by Sebguer » Mon, Apr 29, 2024, 5:29 PM UTC

drno123 wrote: post #13

Still gives incorrect code to the following prompt - the description is correct but not the code. I have yet to find one LLM that gives the correct code. This is the prompt: “Write C code to calculate the sum of matrix elements below the secondary diagonal.“

This feels as much likely to be a prompt problem as a 'bad answer from a model' problem.