161–170 of 1001 posts
"AI Overviews now have 2 billion users every month." "Users"? Or people that get presented with it and ignore it?
Cringe. To get to 2 billion a month they must be counting anyone who sees an AI overview as a user. They should just go ahead and claim the "most quickly adopted product in history" as well.
Earlier quoted context omitted.
I'm not familiar enough with CAD what type of format is it?
It’s not a format, but in my mind it implies designs that are supposed to be functional as opposed to models that are meant for virtual games. It generated a blender script that makes the model.
Pelican riding a bicycle: https://pasteboard.co/CjJ7Xxftljzp.png
The fact that these models can keep getting better at this task given the setup of training is mind-boggling to me.
The ARC puzzles in question: https://arcprize.org/arc-agi/2/
Earlier quoted context omitted.
Why is this particular benchmark important?
Thus far, this is one of the best objective evaluations of real world software engineering...
Earlier quoted context omitted.
It’s not a format, but in my mind it implies designs that are supposed to be functional as opposed to models that are meant for virtual games. It generated a blender script that makes the model.
Did your prompt instruct it to use blender?
> The Gemini app surpasses 650 million users per month, more than 70% of our Cloud customers use our AI, 13 million developers have built with our generative models, and that is just a snippet of the impact we’re seeing Not to be a negative nelly, but these numbers are definitely inflated due to Google literally pushing their AI into everything they can, much like M$. Can't even search google without getting an AI re…
> Gemini app surpasses 650 million users per month Unless these numbers are just lies, I'm not sure how this is "pushing their AI into everything they can". Especially on iOS where every user is someone who went to App Store and downloaded it. Admittedly on Android, Gemini is preinstalled these days but it's still a choice that users are making to go there rather than being an existing product they happen to user oth…
Earlier quoted context omitted.
How can you be sure that your benchmark is meaningful and well designed? Is the only thing that prevents a benchmark from being meaningful publicity?
I didn't tell you what you should think about the model. All I said is that you should have your own benchmark. I think my benchmark is well designed. It's well designed because it's a generalization of a problem I've consistently had with LLMs on my code. Insofar that it encapsulates my coding preferences and communication style, that's the proper benchmark for me.
I've been thinking about making/publishing a new eval - if it's not public, presumably LLMs would never get better at them. But is your fear that generally speaking, LLMs tend to (I don't want to say cheat but) overfit on known problems, but then do (generally speaking) poorly on anything they haven't seen?
Thanks