Live data from Hacker News

Google's Bard shows big leap on LLM performance leaderboard

twitter.com

61–70 of 96 posts

Re: Google's Bard shows big leap on LLM performance leaderboard

#61

Earlier quoted context omitted.

Lazy??? What does it mean for a model to be lazy?

Does not completely answer the question, all aspects of a question, or has a response otherwise cut off. This was common for gpt4 and coding questions it would simply stop responding in the middl

ChatGPT, is that you?

Re: Google's Bard shows big leap on LLM performance leaderboard

#62
post #8

I'm curious about how the benchmark is done. I suspect it can be improved in order to represent user's / usability expectations. I gave Bard a go, after seeing Jeff Dean's tweet. It's just as frustrating as it was, compared to GPT-4. It's simply off the question and unable to realize it's off. I asked it to generate a chart and 3 times it came back with "here's a chart" with no chart, finally saying it doesn't have t…

Bard is free, while GPT-4 is not, so it doesn’t seem like a totally fair comparison. Also what a wild comparison, because afaik chat gpt can’t make charts either.

It absolutely can. Try asking ChatGPT4 something like "Please draw a chart of the population of Switzerland over the past 20 years".

It will write Python code to use matplotlib and the resulting chart will appear in the chat. I'd show an example but their sharing feature doesn't work with images for some reason.

Re: Google's Bard shows big leap on LLM performance leaderboard

#63

Earlier quoted context omitted.

It's actually better. All the people claiming unannounced updates make OpenAI model significantly worse were misled by their own lying eyes. It's so hard to believe, I know, but it's true.

It isn’t, because performance isn’t scalar. It seems to be a more preferable chatbot in this arena. It is objectively less capable for many other domain specific tasks.

How does the arena work? ;)

Re: Google's Bard shows big leap on LLM performance leaderboard

#65

Earlier quoted context omitted.

It could be. Bard is clearly not the best model Google has and the only reason not to serve the better models is inference cost.

Google is fighting for its life here. They are not worried about cost.

Then what's your explanation for them not deploying Gemini?

It's either to expensive or they just don't think it's that important to have a competitive, accessible model out there (which is a valid stance imo).

Re: Google's Bard shows big leap on LLM performance leaderboard

#66

Earlier quoted context omitted.

No clue, but it wouldn't surprised me if they identified inputs and training that weren't actually helping. Just as human beings aren't necessarily more helpful by having more varied input, the same seems to apply to LLMs. The interesting thing is it still has a 128k context length. This is awesome because GPT became way more useful to me once it reached this level of context.

Just a caveat, this is 128k context length for ingestion, I believe the output is still constrained to just 4k tokens.

Yes, this is true. Nevertheless, that made a huge difference for my workflow. I haven't had to think about exceeding the total context in a long time.

Re: Google's Bard shows big leap on LLM performance leaderboard

#67

From all free LLMs I find bard to be most useful. Chatgpt 3.5 is not even close and it lazy.

Lazy??? What does it mean for a model to be lazy?

Q: Please write me a function that makes a couple of text edits and a button.

A: Certainly! Here is a function that makes a couple of text edits and a button.

    function renderForm() {
        var edit = document.createElement('input');
        // The rest of the code is similar.
    }
Post reply on HN