Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4.5 or GPT-5 being tested on LMSYS?

rentry.co

Post reply on HN
Print view View original

331–340 of 380 posts

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#331

Post by arcastroe »

Woah, gpt2-chatbot is the ONLY llm that has ever answered this question correctly.

> How do I reduce an instance of the subset sum problem containing positive and negative integers, to an instance that contains positive integers only?

There is a very simple transformation. But if you google for this, almost all of the math-exchange answers are either flat out incorrect, or technically correct but overly complex.

This was my go-to question to test llms, and every single LLM that I have tried before this would repeat the same incorrect answer that you would find on a forum.

gpt2-chatbot answered it correctly on its first attempt.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#332

Post by Mkengine »

skissane wrote: post #140

gpt2-chatbot is not the only "mystery model" on LMSYS. Another is "deluxe-chat". When asked about it in October last year, LMSYS replied [0] "It is an experiment we are running currently. More details will be revealed later" One distinguishing feature of "deluxe-chat": although it gives high quality answers, it is very slow, so slow that the arena displays a warning whenever it is chosen as one of the competitors [0]…

I looked 3 times through the list, I can't find a "deluxe-chat" there, how do I select it?

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#333

Post by zimmund »

peheje wrote: post #330

Opening this page caused my RAM usage of Chrome to skyrocket. Amount of processes doubled. When looking at Chrome task manager (Windows shortcut Shift+Esc) list, sub-frames popped up I didn't recognize such as: https://adnxs.com/ https://adsrvr.org/ https://btloader.com/ https://doubleclick.net/ https://gumgum.com/ https://openx.net/ https://smartadserver.com/ And many more. Each taking up RAM. In incognito mode / wi…

The site seems to use gradio (which from what I've experienced in the past is quite slow).

All those domains can be avoided just using an ad blocker, I suggest ublock origin.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#335

Post by Red_Leaves_Flyy »

peheje wrote: post #330

Opening this page caused my RAM usage of Chrome to skyrocket. Amount of processes doubled. When looking at Chrome task manager (Windows shortcut Shift+Esc) list, sub-frames popped up I didn't recognize such as: https://adnxs.com/ https://adsrvr.org/ https://btloader.com/ https://doubleclick.net/ https://gumgum.com/ https://openx.net/ https://smartadserver.com/ And many more. Each taking up RAM. In incognito mode / wi…

You don’t recognize double click?

You aren’t running a blocker?

You know how to get this information while answering yes to the above two questions?

Weird.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#336

Post by hombre_fatal »

jprete wrote: post #84
hombre_fatal wrote: post #37

Man, its knowledge is insane. I run a dying forum. I first prompted with "Who is at ?" and it gave me a very endearing, weirdly knowledgeable bio of myself and my contributions to the forum including various innovations I made in the space back in the day. It summarized my role on my own forum better than I could have ever written it. And then I asked "who are other notable users at " and it gave me a list of some mo…

I wonder if it's RAGing? Dying Internet forums are way, way into the tail of the data.

What database would it RAG from? The long tail in the model's data is also the long tail of any data. There are no google searches that have the information it provided about my forum nor is the info neatly collated anywhere on my own forum.

Its knowledge about my forum isn't only obscure, it's also an itemization of obscure events over time to draw obscure conclusions that only a historian of my forum would know. That's what's so impressive.

Granted, my forum was once the largest forum of its genre though that was almost 15 years ago, so it's not some dead proboards forum.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#337

Post by [deleted] »

SOLAR_FIELDS wrote: post #280
Attummm wrote: post #249

Earlier quoted context omitted.

If you had used GPT-4 from the beginning, the quality of the responses would have been incredibly high. It also took 3 minutes to receive a full response. And prompt engineering tricks could get you wildly different outputs to a prompt. Using the 3xx ChatGPT4 model from the API doesn't hold a candle to the responses from back then.

I hear this somewhat often from people (less so nowadays) but before and after prompt examples are never provided. Do you have some example responses saved from the olden days, by chance? It would be quite easy to demonstrate your point if you did.

[deleted]

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#338

Post by Filligree »

parineum wrote: post #324
Workaccount2 wrote: post #298

Earlier quoted context omitted.

Usually when I encounter sentiment like this it is because they only have used 3.5 (evidently not the case here) or that their prompting is terrible/misguided. When I show a lot of people GPT4 or Claude, some percentage of them jump right to "What year did Nixon get elected?" or "How tall is Barack Obama?" and then kind of shrug with a "Yeah, Siri could do that ten years ago" take. Beyond that you have people who pro…

> or that their prompting is terrible/misguided. This is the "You're not using it right" defense. It's an LLM, it's supposed to understand human language queries. I shouldn't have to speak LLM to speak to an LLM.

That is not the reality today. If you want good results from an LLM, then you do need to speak LLM. Just because they appear to speak English doesn't mean they act like a human would.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#339

Post by vineyardmike »

Eisenstein wrote: post #315
vineyardmike wrote: post #313

Earlier quoted context omitted.

That said, it is pretty SV behavior to have one of your companies pay the other. A subtle wealth transfer from OpenAI/Microsoft to Reddit (and tbh other VC backed flailing companies) would totally make sense. VC companies for years have been parroting “data is the new oil” while burning VC money like actual oil. Crazy to think that the latest VC backed companies with even more overhyped valuations suddenly need these…

> A subtle wealth transfer from OpenAI/Microsoft to Reddit (and tbh other VC backed flailing companies) would totally make sense. That's the confusing part -- the person I responded to posited that they didn't pay reddit and thus couldn't use the data which is the only scenario that doesn't make sense to me.

I suppose a "data transfer" from Reddit to OAI would be valuable for SamA too? Still a transfer of value from one hand to the other, while others (eg. Google) have to pay.

That said, I wouldn't be surprised if they pay now. They can't get away with scraping as easily now that they are better-known and commercially incentivized.

Re: GPT-4.5 or GPT-5 being tested on LMSYS?

#340

Post by titaniumtown »

peheje wrote: post #330

Opening this page caused my RAM usage of Chrome to skyrocket. Amount of processes doubled. When looking at Chrome task manager (Windows shortcut Shift+Esc) list, sub-frames popped up I didn't recognize such as: https://adnxs.com/ https://adsrvr.org/ https://btloader.com/ https://doubleclick.net/ https://gumgum.com/ https://openx.net/ https://smartadserver.com/ And many more. Each taking up RAM. In incognito mode / wi…

this is why you use Firefox+ublock origin!
Post reply on HN