Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI GPT-4 vs. Groq Mistral-8x7B

serpapi.com

Post reply on HN
Print view View original

81–90 of 139 posts

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#81

Post by IshanMi »

AlchemistCamp wrote: post #67
shanehoban wrote: post #64

Earlier quoted context omitted.

This is a valid point, but we are still in the early stages of AI/LLMs, so one would expect the speed and efficiency to improve drastically (perhaps accuracy too) over the coming years. At least AI & LLMs have large scale practical applications as opposed to crypto (IMO).

AI is a lot older than blockchain. There were full-fledged neural networks in the 40s and the perceptron was implemented in hardware in the 50s.

It's also interesting to think that IBM released an 8-trillion parameter model back in the 1980s [0]. Granted it was an n-gram model so it's not exactly an apples-to-apples comparison with today's models, but still, quite crazy to think about.

[0]: https://aclanthology.org/J92-4003.pdf

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#82

Post by lm28469 »

rafaelero wrote: post #74
dns_snek wrote: post #56

For all the posturing and crypto hate on HN, we're entering a world where it's socially acceptable to use 1000W of computing power and 5 seconds of inference time to parse a tiny HTML fragment which would take microseconds with traditional methods - and people are cheering about it. Time for some self-reflection? That's not very green.

What a ridiculous complaint. Energy efficiency won't remain static, and even if it were, it's not up to you to decide how to best leverage the available electricity.

> it's not up to you to decide

Unless you live in a dictatorship it's definitely up to us to decide... Otherwise you leave your voice to the top 0.0001% business owners and expect them to work for your good and not for their own interests

Also read about the rebound effect. Planes are twice as efficient as they were 100 years ago yet they pollute infinitely more as a whole.

There is nothing ridiculous about the comment you're replying to

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#83

Post by Jensson »

ch_sm wrote: post #72
simion314 wrote: post #43

Earlier quoted context omitted.

The problem is asking for facts, LLM are not a database so they know stuff but it is compressed so expect wrong facts, wrong names, dates, wrong anything. We will need an LLM as a front end then it will generate a query to fetch the facts from the internet or a database , then maybe format the facts for your consumption.

That‘s exactly it. It‘s ok for LLMs to not know everything, because they _should_ have a means to look up information. What are some projects where this obvious approach is implemented/tried?

But then you need an LLM that can separate between grammar and facts. Current LLMs doesn't know the difference, that is the main source to these issues, these models treat facts like grammar and that worked well enough to excite people but probably wont get us to a good state.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#84

Post by onion2k »

retrac98 wrote: post #3

There are so many applications for LLMs where having a perfect score is much more important than speed, because getting it wrong is so expensive, damaging, or time consuming to resolve for an organisation.

All the places I see AI being applicable to my work don't require a perfect score, and a threshold is actually much more useful, especially where multiple factors come together to make evaluation to a single value hard.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#85

Post by varjag »

AlchemistCamp wrote: post #67
shanehoban wrote: post #64

Earlier quoted context omitted.

This is a valid point, but we are still in the early stages of AI/LLMs, so one would expect the speed and efficiency to improve drastically (perhaps accuracy too) over the coming years. At least AI & LLMs have large scale practical applications as opposed to crypto (IMO).

AI is a lot older than blockchain. There were full-fledged neural networks in the 40s and the perceptron was implemented in hardware in the 50s.

I wouldn't call the early McCulloch & Pitts work quite "full-fledged". Also backpropagation, essential for multi level perceptrons was not a thing until 1980s.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#87

Post by HarHarVeryFunny »

whiplash451 wrote: post #52
cornedor wrote: post #21

Earlier quoted context omitted.

According to Sam Altman in a podcast with Lex Fridman this week, there is no real indication that it will be dropped this year. They will release a new model, but it might not be GPT-5

Which is an indication of nothing. In which world would Sam A. drop any kind of info about such a sensitive topic? If anything, this could just be deception before a massive drop.

Could also be resetting expectations for people who've been expecting GPT-5 (or just GPT-4.5) sooner - been a year now since GPT-4 was released.

The other odd thing from Altman was saying that GPT-4 sucks.

I think the context for both announcements is the recent release of Anthropic's Claude-3, which in it's largest "Opus" form beats GPT-4 across the board in benchmarks.

I personally think OpenAI/Altman is a bit scared that any moat/lead they had has disappeared and they are now being out-competed by Anthropic (Claude). Remember that Anthropic as a company was only formed (by core members of the OpenAI LLM team) at the same time as GPT-3 was released, so in same time it took OpenAI to go from GPT-3 to GPT-4, Anthropic have gone from nothing -> Claude-1 -> Claude-2 -> Claude-3 which beats GPT-4 !!

Anthropic have also had quite a bit of success attracting corporate business, quite a bit of which is more long-term in nature (sharing details of expected future model capabilities so that partners can target those).

So, I think OpenAI is running a bit scared, and I'd interpret this non-announcement of some model (4.5 or 5) "coming soonish" to be them just waving the flag and saying "we'll be back on top soon", which they presumably will be, briefly, when their next release(s) do come out. Altman's odd "GPT-4 sucks" statement might be meant to downplay Claude-3 "Opus" which beats it.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#88

Post by Eisenstein »

spaniard89277 wrote: post #16
retrac98 wrote: post #3

There are so many applications for LLMs where having a perfect score is much more important than speed, because getting it wrong is so expensive, damaging, or time consuming to resolve for an organisation.

I've tried to apply it to parsing HTML as this article into a pretty long pipeline. I'm using DeepInfra with Mistral 8x7B and I'm still unsure if I'm going to use for production. The problem I'm finding is that the time I wanted to save mantaining selectors and the like is time that I'm spending writing wrapper code and dealing with the mistakes it makes. Some are OK and can deal with them, others are pretty annoying…

Have you tried swapping Mistral 8x7B with either command-r 34B, Qwen 1.5 70B, or miqu 70B? Those are all superior in my experience, though suited for slightly different tasks, so experimentation is needed.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#89

Post by greenavocado »

m348e912 wrote: post #49
Drakim wrote: post #22

Earlier quoted context omitted.

If I ask an LLM a very complex and specific question 500 times, if it just doesn't know the facts you'll still get the wrong answer 500 times. That's understandable. The real problem is when the AI lies/hallucinates another answer with confidence instead of saying "I don't know".

The weird problem is with LLM hallucinations is that it usually will acknowledge its mistake and correct itself if you call it out. My question is why can't LLMs included a sub-routine to check itself before answering. Simply asking itself something like "this answer may not be correct, are you sure you're right?"

In my experience the corrections can be additional hallucinations one after another after pointing out inaccuracies even multiple times in a row.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#90

Post by lukeschantz »

IshanMi wrote: post #81
AlchemistCamp wrote: post #67

Earlier quoted context omitted.

AI is a lot older than blockchain. There were full-fledged neural networks in the 40s and the perceptron was implemented in hardware in the 50s.

It's also interesting to think that IBM released an 8-trillion parameter model back in the 1980s [0]. Granted it was an n-gram model so it's not exactly an apples-to-apples comparison with today's models, but still, quite crazy to think about. [0]: https://aclanthology.org/J92-4003.pdf

Interesting to see Robert Mercer the former CEO of Renaissance Technology is one of the authors on that paper. He is a former IBMer. If his name is unfamiliar he is a reclusive character who was a major funder of Breitbart, Cambridge Analytica and the Republican candidate in the 2016 presidential election.
Post reply on HN