Live data from Hacker News

Gemini Pro vs. GPT3.5

klu.ai

1–7 of 7 posts

Re: Gemini Pro vs. GPT3.5

#3
This "vibe" check that it's even better than GPT-4 Turbo is not what its Elo rating shows on the Chatbot Arena based on not 1 but thousands of user votes. GPT-4 (Turbo) is in a league of its own still.

Re: Gemini Pro vs. GPT3.5

#4
post #3

This "vibe" check that it's even better than GPT-4 Turbo is not what its Elo rating shows on the Chatbot Arena based on not 1 but thousands of user votes. GPT-4 (Turbo) is in a league of its own still.

The vibe check is for pro tho. I want to see how ultra is benchmarked.

Re: Gemini Pro vs. GPT3.5

#5
post #3

This "vibe" check that it's even better than GPT-4 Turbo is not what its Elo rating shows on the Chatbot Arena based on not 1 but thousands of user votes. GPT-4 (Turbo) is in a league of its own still.

By its nature, that site isn't very representative of how the models perform in real-world use.

Re: Gemini Pro vs. GPT3.5

#6
post #5
post #3

This "vibe" check that it's even better than GPT-4 Turbo is not what its Elo rating shows on the Chatbot Arena based on not 1 but thousands of user votes. GPT-4 (Turbo) is in a league of its own still.

By its nature, that site isn't very representative of how the models perform in real-world use.

The ELO leaderboard you mean?

Re: Gemini Pro vs. GPT3.5

#7
post #5
post #3

This "vibe" check that it's even better than GPT-4 Turbo is not what its Elo rating shows on the Chatbot Arena based on not 1 but thousands of user votes. GPT-4 (Turbo) is in a league of its own still.

By its nature, that site isn't very representative of how the models perform in real-world use.

That depends on what real world use you're targeting, but unfortunately I'm not aware of anything better than that leaderboard in terms of sample size and model coverage.