Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI GPT-4 vs. Groq Mistral-8x7B

serpapi.com

Post reply on HN
Print view View original

121–130 of 139 posts

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#121

Post by samus »

m348e912 wrote: post #49
Drakim wrote: post #22

Earlier quoted context omitted.

If I ask an LLM a very complex and specific question 500 times, if it just doesn't know the facts you'll still get the wrong answer 500 times. That's understandable. The real problem is when the AI lies/hallucinates another answer with confidence instead of saying "I don't know".

The weird problem is with LLM hallucinations is that it usually will acknowledge its mistake and correct itself if you call it out. My question is why can't LLMs included a sub-routine to check itself before answering. Simply asking itself something like "this answer may not be correct, are you sure you're right?"

They simply don't work that way. You are asking it for an answer, it will give you one since all it can do is extrapolate from its training data.

Good prompting and certain adjustment to the text generation parameters might help prevent hallucinations, but it's not an exact science since it depends on how it was trained. Also, an LLMs training data frankly said contains a lot of bulls*t.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#122

Post by samus »

ZitchDog wrote: post #57
m348e912 wrote: post #49

Earlier quoted context omitted.

The weird problem is with LLM hallucinations is that it usually will acknowledge its mistake and correct itself if you call it out. My question is why can't LLMs included a sub-routine to check itself before answering. Simply asking itself something like "this answer may not be correct, are you sure you're right?"

The problem is that if you call it out, it will frequently change its answer, even if it was correct. LLMs currently lack chutzpa.

They definitely stand their ground if they were aligned to do so.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#123

Post by [deleted] »

ch_sm wrote: post #72
simion314 wrote: post #43

Earlier quoted context omitted.

The problem is asking for facts, LLM are not a database so they know stuff but it is compressed so expect wrong facts, wrong names, dates, wrong anything. We will need an LLM as a front end then it will generate a query to fetch the facts from the internet or a database , then maybe format the facts for your consumption.

That‘s exactly it. It‘s ok for LLMs to not know everything, because they _should_ have a means to look up information. What are some projects where this obvious approach is implemented/tried?

[deleted]

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#124

Post by samus »

simion314 wrote: post #43
Drakim wrote: post #22

Earlier quoted context omitted.

If I ask an LLM a very complex and specific question 500 times, if it just doesn't know the facts you'll still get the wrong answer 500 times. That's understandable. The real problem is when the AI lies/hallucinates another answer with confidence instead of saying "I don't know".

The problem is asking for facts, LLM are not a database so they know stuff but it is compressed so expect wrong facts, wrong names, dates, wrong anything. We will need an LLM as a front end then it will generate a query to fetch the facts from the internet or a database , then maybe format the facts for your consumption.

This is called Retrieval Augmented Generation (RAG). The LLM driver recognizes a query, it gets send to a vector database or to an external system (could be another LLM...) and the answer is placed in the context. It's a common strategy to work around their limited context length, but it tends to be brittle. Look for survey papers.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#125

Post by samus »

rybosome wrote: post #93
RUnconcerned wrote: post #26

Earlier quoted context omitted.

Personally when I am parsing structured data I prefer to use parsers that won't hallucinate data but that's just me. Also, don't parse HTML with regular expressions.

Generally I agree with your point, but there is some value in a parser that doesn’t have to be updated when the underlying HTML changes. Whether or not this benefit outweighs the significant problems (cost, speed, accuracy and determinism) is up to the use case. For most use cases I can think of, the speed and accuracy of an actual parser would be preferable. However, in situations where one is parsing highly dynamic…

You could employ an LLM to give you updated queries when the format changes. This is something where they should shine. And you get something that you can audit and exhaustively test.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#126

Post by samus »

infecto wrote: post #103

This test is interesting from a general high level metric/test but overall the way they are extracting data using a LLM is suboptimal so I don't think the takeaway means much. You could extract this type of data using a low-end model like 8x7B with a high degree of accuracy.

The better way would be to ask it to generate a program that uses CSS selectors to parse the HTML.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#127

Post by samus »

crowdyriver wrote: post #86

There's lots of comments here about how stupid is to parse html using llms. Have you ever had to scrape multiple sites with variadic html?

The example here has HTML with a somewhat fixed format. It would indeed have been better to have samples with different format and aiming for a low error rate.

If you are scraping a limited amount of sites, you could for each site ask the LLM for parsing code from some samples, review that, and move on.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#128

Post by samus »

delegate wrote: post #71
dns_snek wrote: post #56

For all the posturing and crypto hate on HN, we're entering a world where it's socially acceptable to use 1000W of computing power and 5 seconds of inference time to parse a tiny HTML fragment which would take microseconds with traditional methods - and people are cheering about it. Time for some self-reflection? That's not very green.

Crypto energy requirements go up as the currency gets more traction. TFA shows that groq is many times faster than GPT-4. Up to 18x groq claims. Faster means less energy. So I think it's just a matter of time until these things become ridiculously power efficient (eg run on phones in sub second times)

It's still a monstrosity compared to a traditional parser. You can even be fancy and use complex parsers that backtrack and can deal with mildly context-sensitive languages (as required for HTML, XML, and many programmin languages), and you'd still be more efficient.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#129

Post by Drakim »

samus wrote: post #122
ZitchDog wrote: post #57

Earlier quoted context omitted.

The problem is that if you call it out, it will frequently change its answer, even if it was correct. LLMs currently lack chutzpa.

They definitely stand their ground if they were aligned to do so.

But then they stand their ground when wrong too.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#130

Post by samus »

varjag wrote: post #85
AlchemistCamp wrote: post #67

Earlier quoted context omitted.

AI is a lot older than blockchain. There were full-fledged neural networks in the 40s and the perceptron was implemented in hardware in the 50s.

I wouldn't call the early McCulloch & Pitts work quite "full-fledged". Also backpropagation, essential for multi level perceptrons was not a thing until 1980s.

Backprop is just applied calculus. People simply didn't think about using it for neuronal networks yet.
Post reply on HN