What is it about LLMs that brings otherwise rational people to become bumbling sycophants??
Because they need to get with the consensus as a baseline and extend from there, otherwise they may look behind
GPT-4.5 or GPT-5 being tested on LMSYS?
131–140 of 380 posts
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#132Man, its knowledge is insane. I run a dying forum. I first prompted with "Who is at ?" and it gave me a very endearing, weirdly knowledgeable bio of myself and my contributions to the forum including various innovations I made in the space back in the day. It summarized my role on my own forum better than I could have ever written it. And then I asked "who are other notable users at " and it gave me a list of some mo…
If this isn’t Google, google stock may really go down hard
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#133What is it about LLMs that brings otherwise rational people to become bumbling sycophants??
Because LLMs are very impressive invention, truly something new we haven't experienced in a long while.
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#134For anyone wondering how to access Go to - https://chat.lmsys.org/ Select Direct Chat Select gpt2-chatbot Other twitter mentions - https://twitter.com/kosenjuu/status/1784952955294421215
[deleted]
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#135I'm impressed. I gave the same prompt to opus, gpt-4, and this model. I'm very impressed with the quality. I feel like it addresses my ask better than the other 2 models. GPT2-Chatbot: https://pastebin.com/vpYvTf3T Claude: https://pastebin.com/SzNbAaKP GPT-4: https://pastebin.com/D60fjEVR Prompt: I am a senate aid, my political affliation does not matter. My goal is to once and for all fix the American healthcare sys…
They all did pretty well tbh. GPT2 didnt talk about moving away from fee for service which I think the evidence shows is the best idea, the other 2 did. GPT2 did have some other good ideas that the others didnt touch on though.
I agree, all 3 were great viable answers to my question. But Claude and GPT-4 felt a lot more like a regurgitation of the same suggestions people have proposed for years (ie, their training material). The GPT2, while similar felt more like it tried to approach the question from first principals. IE, it laid out a set of root causes, and reasoned a solution from there, which was subtly my primary ask.
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#136I find it notable that it's not GPT-2. It's GPT2.
The old models also had only "gpt2"/"gpt2-large" and so on as "identifiers"
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#137Prompt: code up an analog clock in html/js/css. make sure the clock is ticking exactly on the second change. second hand red. other hands black. all 12 hours marked with numbers. ChatGPT-4 Results: https://jsbin.com/giyurulajo/edit?html,css,js,output GPT2-Chatbot Results: https://jsbin.com/dacenalala/2/edit?html,css,js,output Claude3 Opus Results: https://jsbin.com/yifarinobo/edit?html,css,js,output None is correct.…
damn these bots must have dementia
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#138This looks like gpt4 using API I compared outputs and they are very very similar in structure of the response and the details.
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#139Plot twist! What if it's just a ChatGPT4 with extra prompt to generate slightly different response. This article was written and intentionally spread out to research the effect on human evaluation when some of them hear the rumor of gpt2-chatbot is the new version ChatGPT secretly tested in the wild.
I don't think a magical prompt is suddenly going to make any current public model draw an ASCII unicorn like this thing does. (besides, it already leaks a system prompt which seems very basic)
Re: GPT-4.5 or GPT-5 being tested on LMSYS?
#140gpt2-chatbot is not the only "mystery model" on LMSYS. Another is "deluxe-chat".
When asked about it in October last year, LMSYS replied [0] "It is an experiment we are running currently. More details will be revealed later"
One distinguishing feature of "deluxe-chat": although it gives high quality answers, it is very slow, so slow that the arena displays a warning whenever it is chosen as one of the competitors