Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.5 or GPT-5 being tested on LMSYS?

rentry.co

Post reply on HN
Print view View original

151–160 of 380 posts

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#152

Post by krainboltgreene »

harryp_peng wrote: post #150
hehdhdjehehegwv wrote: post #143

Earlier quoted context omitted.

The level of competition right now is so crazy, if it ISN’T a new version of ChatGPT then OpenAI is in trouble. GPT-4 isn’t significantly better than Llama 3. Sure, it’s better, but the margins are evaporating fast. (I’d mention Gemini…but why bother? IMHO, this is now OpenAI the company vs Open models)

This really is the fastest growing technology of all time. Do you feel the curve? I remember Mixtral8x7b dominating for months; I expected data bricks to do the same! but it was washed out of existence in days, with 8x22b, llama3, gemini1.5... WOW.

I must be missing something because the output from two years ago feels exactly the same as the output now. Any comment saying the output is significantly better can be equally pared with a comment saying the output is terrible/censored/"nerfed".

How do you see "fastest growing technology of all time" and I don't? I know that I keep very up to date with this stuff, so it's not that I'm unaware of things.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#153

Post by harryp_peng »

throwup238 wrote: post #46
steve_adams_86 wrote: post #10

The results of this LLM are consistently far better than any other that I choose. I asked ‘what is the most efficient approach to building a led grow light with off-the-shelf parts?’ and its response was incredible. Very much in line with how I’ve done it in the past after weeks of research, trial and error, and feedback from people. The other LLMs gave mostly reasonable yet sparse and incomplete answers. It also opt…

I'm asking it about how to make turbine blades for a high bypass turbofan engine and it's giving very good answers, including math and some very esoteric material science knowledge. Way past the point where the knowledge can be easily checked for hallucinations without digging into literature including journal papers and using the math to build some simulations. I don't even have to prompt it much, I just keep saying…

You know at one point we wouldn't be able to benchmark them, due to the sheer complexity of the test required. I.e. if you are testing a model on maths, the problem will have to be extremely difficult to even consider a 'hustle' for the LLM; it would then take you a day to work out the solution yourself.

See where it's getting at? When humans are no longer on the same spectrum as LLMs, that's probably the definition of AGI.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#154

Post by monday40 »

Hmm... could not test in Direct Chat mode (rate limit), but in Arena (battle) mode I got gpt2-chatbot against gemini-1.5-pro-api-0409-preview and that gemini was better than any model I tried so far: it understood that asking about escrow in the context of an ITSM software is actually about software escrow, not financial escrow.

The prompt was: Does HaloITSM have escrow functionality?

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#155

Post by HeatrayEnjoyer »

krainboltgreene wrote: post #152
harryp_peng wrote: post #150

Earlier quoted context omitted.

This really is the fastest growing technology of all time. Do you feel the curve? I remember Mixtral8x7b dominating for months; I expected data bricks to do the same! but it was washed out of existence in days, with 8x22b, llama3, gemini1.5... WOW.

I must be missing something because the output from two years ago feels exactly the same as the output now. Any comment saying the output is significantly better can be equally pared with a comment saying the output is terrible/censored/"nerfed". How do you see "fastest growing technology of all time" and I don't? I know that I keep very up to date with this stuff, so it's not that I'm unaware of things.

The best we had two years ago was GPT-3, which was not even instruction tuned and hallucinated wildly.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#158

Post by kodisha »

laluneodyssee wrote: post #156
kodisha wrote: post #151

@sama on X/Twitter: > i do have a soft spot for gpt2 https://twitter.com/sama/status/178510794366456

that was quickly deleted...

Oh, wow.

People have screenshots on X:

https://pbs.twimg.com/media/GMX9e_3XMAAuvoo?format=jpg&name=...

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#160

Post by cma »

krainboltgreene wrote: post #152
harryp_peng wrote: post #150

Earlier quoted context omitted.

This really is the fastest growing technology of all time. Do you feel the curve? I remember Mixtral8x7b dominating for months; I expected data bricks to do the same! but it was washed out of existence in days, with 8x22b, llama3, gemini1.5... WOW.

I must be missing something because the output from two years ago feels exactly the same as the output now. Any comment saying the output is significantly better can be equally pared with a comment saying the output is terrible/censored/"nerfed". How do you see "fastest growing technology of all time" and I don't? I know that I keep very up to date with this stuff, so it's not that I'm unaware of things.

Are you trying the paid gpt or just free 3.5 chatgpt?
Post reply on HN