Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI GPT-4 vs. Groq Mistral-8x7B

serpapi.com

Post reply on HN
Print view View original

111–120 of 139 posts

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#111

Post by Kostic »

tosh wrote: post #38
YetAnotherNick wrote: post #12

Earlier quoted context omitted.

There's no reason for OpenAI to release the model. They have close to 100% market anyways and releasing GPT-5 likely won't increase the total market as it is a incremental leap. And it's a open secret that most other models used GPT-4 synthetic data for training to come close to it. They would likely wait till any model performs better than GPT 4 for the same price

100%? Claude 3 Opus is in the capability ballpark of GPT-4, GPT-3.5 has alternatives that are cheaper (Claude 3 Haiku) or cheaper and work offline (Qwen 1.5, Mixtral, …).

Is Claude 3 Opus generating more profits and taking considerable amount of customers from OpenAI? I'm not seeing that yet. Granted, I'm in Europe (outside of EU) so I can't pay for Opus but I guess that kinda confirms my statement. GPT4 is still a good product and there are no market pressures to release GPT5.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#112

Post by wruza »

infecto wrote: post #96
wruza wrote: post #60

The prompt, for those interested. I find it pretty underspecified, but maybe that's the point. For example, "Business operating hours" could be expanded a little, because "Closed - Opens at XX" is still non-processable in both cases. You are an expert in Web Scraping, so you are capable to find the information in HTML and label them accordingly. Please return the final result in JSON. Data to scrape: title: Name of t…

This should be higher up. This whole blog post is mostly worthless because the way they are extracting data is less than optimal. Lower end models do not have the attention to complete tasks like this, GPT4Turbo will generally have the capability. But to have an optimal pipeline you should really be splitting up these tasks into individual units. You extract each attribute you want independently and then combine it b…

Also, my (limited) experience with prompts tells that you want to invest more into the “You are” part. I’ll share my understanding, corrections are appreciated.

LLMs aren’t people even in a chat-roleplaying sense. They complete a “document” that can be a plot, a book, a protocol of conversation. The “AI” side in the chat isn’t an LLM itself, it’s a character (and so are you, it completes your “You: …” replies too - that’s where the driver app stops it and allows you to interfere). So everything you put in that header is very important. There are two places where you can do that: right in the chat, as in TFA, or in the “character card” (idk if GPTs have it, no GPT access for me). I found out that properly crafting a character card makes a huge difference and can resolve the whole classes of issues.

Idk what will work best in this case, but I’d start with describing which sort of a bot, how it deals with unclear or incomplete information, how amazing it is (yes, really), its soft/tech skills and problem solving abilities, what other people think of it, their experience and so on. Maybe would add few examples of interactions in a free form. Then in the task message I’d tell it more and specific details about that json.

One more note - at least for 8x7B, the “You are” in the chat is a much weaker instruction than a character card, even if the context is still empty. I low-key believe that’s because it’s a second-class prompt, i.e. the chat document starts with “This is a conversation with a helpful AI bot which yada yada” in… mind, and then in that chat that AI character gets asked to turn into something else, which poisons the setting.

Simply asking the default AI card represents 0.1% of what’s possible and doesn’t give the best results. Prompt Engineering is real.

I have high confidence that I could accomplish this task using a lower end model with a high degree of accuracy.

Same. I think that no matter how good a model is, this prompt just isn’t a professional task statement and leaves too much to decide. It’s a task that you, as a regular human, would hate to receive.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#113

Post by samus »

jojobas wrote: post #35
samus wrote: post #24

Earlier quoted context omitted.

They already have superhuman image classification performance.

Until a single pixel makes a cat a dog or something like that.

Changing a single pixel is usually not enough to confuse convolutional neuronal networks. Even so, human supervision will probably always be quite important.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#114

Post by CaptainFever »

ogogmad wrote: post #104
dns_snek wrote: post #56

For all the posturing and crypto hate on HN, we're entering a world where it's socially acceptable to use 1000W of computing power and 5 seconds of inference time to parse a tiny HTML fragment which would take microseconds with traditional methods - and people are cheering about it. Time for some self-reflection? That's not very green.

You're partially right. It's obvious that the solution is to combine traditional programming with AI, using traditional programming wherever possible because it's greener. Assuming you want things to turn out well in every possible future scenario, your decisions only matter if AGI isn't right around the corner. So assume it isn't right around the corner. Then there's going to be some interesting combining-together o…

> We'll need to charge more for some uses of electricity, to incentivise turning AI into traditional software wherever possible.

I don't understand this. This adds bureaucracy and I don't see why different uses need to be charged differently if they all use energy the same.

In other words, if energy costs X per unit, and an inefficient (AI) software takes 30 units and an efficient (traditional) software takes 10 units, then it is already cheaper to run the efficient software, and thus people are already incentivised to do so. There's no need to charge differently. If one day AI turns out to only need 5 units, turning more efficient, then just charge them for 5X. People will gravitate towards the new, efficient AI software naturally then.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#115

Post by Socnic »

nathan_compton wrote: post #31
retrac98 wrote: post #23

Earlier quoted context omitted.

They'll get better. Humans are far from perfect, and I have no doubt that LLMs will eventually outperform them for non-trivial tasks consistently.

Maybe so, but at this stage I wouldn't be betting a business model on it.

Businesses do bet on imperfect and even criminal models all the time (way before LLMs existed)... they call it cost of doing business when they get it wrong or get caught.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#116

Post by samus »

harryf wrote: post #44
RUnconcerned wrote: post #30

Earlier quoted context omitted.

I too am a big fan of having my computer hallucinate incorrect information.

>> print(“Hello, world!”.ai_reverse()) world, Hello!

Character-level operations are difficult for LLMs. Because of tokenization they don't really "perceive" strings as a list of characters. There are LLMs that ingest bytes, but they are intended to process binary data.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#117

Post by YetAnotherNick »

whiplash451 wrote: post #53
YetAnotherNick wrote: post #12

Earlier quoted context omitted.

There's no reason for OpenAI to release the model. They have close to 100% market anyways and releasing GPT-5 likely won't increase the total market as it is a incremental leap. And it's a open secret that most other models used GPT-4 synthetic data for training to come close to it. They would likely wait till any model performs better than GPT 4 for the same price

The same reasoning would have applied for GPT-3.5. In the hindsight , you can say that it was obviously a good idea to build and ship GPT4. But hindsight is 20/20.

There are few differences. Firstly, GPT-3.5 wasn't ahead of Palm etc. from Google which was published at the same time as GPT-4.

Secondly, GPT-4 increased overall AI market. According to all the sources, interviews and leaks, GPT-5 won't be a big leap over GPT-4 as the model size and training data won't be significantly larger. I doubt GPT-5 would do that. (I could be wrong in my assumption though that GPT-5 would just be a incremental gain).

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#118

Post by YetAnotherNick »

chilmers wrote: post #54
YetAnotherNick wrote: post #12

Earlier quoted context omitted.

There's no reason for OpenAI to release the model. They have close to 100% market anyways and releasing GPT-5 likely won't increase the total market as it is a incremental leap. And it's a open secret that most other models used GPT-4 synthetic data for training to come close to it. They would likely wait till any model performs better than GPT 4 for the same price

By any chance did you used to work in leadership at Nokia or Research in Motion? :-D

Nokia wasn't that ahead in technology and Motion wasn't that ahead in market. GPT-4 is ahead in both.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#119

Post by londons_explore »

osigurdson wrote: post #101
londons_explore wrote: post #62

Earlier quoted context omitted.

While energy remains cheap and human minds remain expensive, it always makes sense to use AI to reduce human effort. If one cares about the environment, a carbon cap/tax is what you should campaign for. Then carbon-based energy sources will be curtailled, energy costs will go up, and AI like this will be encouraged to become more energy efficient or other methods used instead.

It is a nice idea in principle but ends up being a political tool and a tariff on goods and services of your own country. A global and corruption free carbon tax might work but that is impossible to achieve.

The only way it's gonna work is if a bunch of countries get together, agree a carbon cap/tax, and then tell other countries that they need to join the scheme if they want to trade goods with the group.

One way to combat corruption is to ask an international panel of experts to assess how many extra emissions came from non-official sources in each country and reduce next years cap by that amount. Then countries have an incentive to stamp out corruption.

Re: OpenAI GPT-4 vs. Groq Mistral-8x7B

#120

Post by samus »

spaniard89277 wrote: post #16
retrac98 wrote: post #3

There are so many applications for LLMs where having a perfect score is much more important than speed, because getting it wrong is so expensive, damaging, or time consuming to resolve for an organisation.

I've tried to apply it to parsing HTML as this article into a pretty long pipeline. I'm using DeepInfra with Mistral 8x7B and I'm still unsure if I'm going to use for production. The problem I'm finding is that the time I wanted to save mantaining selectors and the like is time that I'm spending writing wrapper code and dealing with the mistakes it makes. Some are OK and can deal with them, others are pretty annoying…

Parsing HTML and tagsoup is IMHO not the right application for LLMs since these are ultimately structured formats. LLM are for NLP tasks, like extracting meaning out of unstructured and ambiguous text. The computational cost of an LLM chewing through even moderately-sized document can be more efficiently spent on sophisticated parser technologies that have been around for decades, which can also to a degree deal with ambiguous and irregular grammars. LLMs should be able to help you write those.
Post reply on HN