ChatGPT for Teams

Post by Doubtme » Thu, Jan 11, 2024, 6:37 AM UTC

Post by MacsHeadroom » Thu, Jan 11, 2024, 6:43 AM UTC

icelancer wrote: post #89

benreesman wrote: post #84

Earlier quoted context omitted.

Keep in mind that modern quantitative approaches to LLM evaluation have been effectively co-designed with the rise of OpenAI, and folks like Ravenwolf routinely disagree with the leaderboards. There's also very little if any credible literature on what constitutes statistically significant on MMLU or whatever. There's such a massive vested interest from so many parties (the YC ecosystem is invested in Sam, MSFT is in…

> It's easy and cheap to just try both these days, don't take my word for which one is better. I literally use 8x-7b on my on-prem GPU cluster and have several fine tunes of 7b (which I said in the previous post). I've used mistral-medium. GPT-4-turbo is better than them all on all benchmarks, human preference, and anything that isn't biased vibes. My opinion - such that it is - is that GPT-4-turbo is by far the best…

Post by benreesman » Thu, Jan 11, 2024, 6:44 AM UTC

Demiurge wrote: post #83

walteweiss wrote: post #79

Earlier quoted context omitted.

Oh thank you, I was reading and none of that made any sense to me. I thought it could be a presentation of some dumb AI output. Now I see I’m not alone.

That gave me an idea, here is what I got from Copilot: You have set up your system to run different AI models and compare their performance using a text editor. You are using Mixtral-8x7, a high-quality open-source model developed by Mistral AI, Dolphin, an emulator for Nintendo video games, 3.5-Turbo, a customized version of GPT-3.5, a powerful natural language model, and 4-Series Preview, a new version of the BMW s…

Post by leo150 » Thu, Jan 11, 2024, 6:52 AM UTC

icelancer wrote: post #71

benreesman wrote: post #66

I’ve got my stuff rigged to hit mixtral-8x7, and dolphin locally, and 3.5-turbo, and the 4-series preview all with easy comparison in emacs and stuff, and in fairness the 4.5-preview is starting to show some edge on 8x7 that had been a toss-up even two weeks ago. I’m still on the mistral-medium waiting list. Until I realized Perplexity will give you a decent amount of Mistral Medium for free through their partnership…

> Mistral Medium destroys the 4.5 preview. On what metrics? LMSys shows it does well but 4-Turbo is still leading the field by a wide margin. I am using 8x-7b internally for a lot of things and Mistral-7b fine-tunes for other specific applications. They're both excellent. But neither can touch GPT-4-turbo (preview) for wide-ranging needs or the strongest reasoning requirements. https://huggingface.co/spaces/lmsys/cha…

Post by danielbln » Thu, Jan 11, 2024, 6:54 AM UTC

MeImCounting wrote: post #93

Demiurge wrote: post #83

Earlier quoted context omitted.

That gave me an idea, here is what I got from Copilot: You have set up your system to run different AI models and compare their performance using a text editor. You are using Mixtral-8x7, a high-quality open-source model developed by Mistral AI, Dolphin, an emulator for Nintendo video games, 3.5-Turbo, a customized version of GPT-3.5, a powerful natural language model, and 4-Series Preview, a new version of the BMW s…

I dont think it has anything to do with a BMW sports coupe. What I am confused about though is it seems like the parent is mentioning models beyond the GPT4 instance I currently have access to. I checked their twitter and I have seen no anouncement for any 4.5 or 4 series previews. Is this just available to people using the API or did I miss something?