An In-depth Look at Gemini's Language Abilities • Hacker News Board

Re: An In-depth Look at Gemini's Language Abilities

#71

Post by unstuck3958 » Wed, Dec 20, 2023, 3:37 AM UTC

coder543 wrote: post #45

unstuck3958 wrote: post #2

It's incredible how accurate the Chatbot Arena Leaderboard [0] is at predicting model performance compared to benchmarks (which can and are being gamed, see all the 7B models on HF leaderboard) [0]: https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

I wish that Arena included a few more "interesting" models like the new Phi-2 model and the current tinyllama model, which are trying to push the limits on small models. Solar-10.7B is another interesting model that seems to be missing, but I just learned about it yesterday, and it seems to have come out a week ago, so maybe it's too new. Solar supposedly outperforms Mixtral-8x7B with a fraction of the total paramete…

> like the new Phi-2 model

Phi-2 isn't fine tuned for instruction following yet.

Re: An In-depth Look at Gemini's Language Abilities

#72

Post by czl » Wed, Dec 20, 2023, 5:57 PM UTC

myko wrote: post #25

we_love_idf wrote: post #22

I don't understand why people keep falling for Google's ad campaign. Google have its lead in AI playing video games and board games. It is cool, entertaining and all that jazz. But OpenAI and MS are the real leaders in real AI.

https://arxiv.org/abs/1706.03762 https://research.google/pubs/attention-is-all-you-need/ let's not forget where this breakthrough came from, i wouldn't count Google out

I hope you are correct yet Xerox and Kodak paid for plenty of breakthroughs, did they not? Google faces a similar "innovator's dilemma" challenge: successful companies often struggle to embrace new technologies or business models that could disrupt their existing, successful products.

Re: An In-depth Look at Gemini's Language Abilities

#73

Post by vitorgrs » Sat, Dec 23, 2023, 10:15 PM UTC

sidibe wrote: post #36

we_love_idf wrote: post #22

I don't understand why people keep falling for Google's ad campaign. Google have its lead in AI playing video games and board games. It is cool, entertaining and all that jazz. But OpenAI and MS are the real leaders in real AI.

Even if you don't think Google doesn't have a talent or product chops to be leaders in AI, Google can do things cheaper than others because of their infrastructure. When they do release something useful they'll probably be able to offer it free and force it on people on the most visited pages/most used browser. Surprised how many people think having a years head start means OpenAI and Microsoft are going to always be…

I don't even think Gemini Ultra will be free. They said it will be offered on "Bard Advanced", so not standard Bard... Which makes me thinking that it might be paid.