Earlier quoted context omitted.
[flagged]
[flagged]
Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
Meta got caught gaming AI benchmarks
151–160 of 171 posts
Re: Meta got caught gaming AI benchmarks
#152Earlier quoted context omitted.
[flagged]
[flagged]
Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
Re: Meta got caught gaming AI benchmarks
#153This feels like AI deniers grasping at straws. You have to prove the claim that "friendly" models do better in head to head user ratings. You also have to adequately define your claim which hasn't been done. And then you have to prove that by whatever definition of "friendliness" you've constructed made a significant difference in the benchmark.
What I suspect will happen is that someone will just tell the LLM to "be curt" that may cause it's score to drop and people will unthinkingly take that to mean the above nonsensical claims are true.
Re: Meta got caught gaming AI benchmarks
#154Re: Meta got caught gaming AI benchmarks
#155wow, is that real?
Re: Meta got caught gaming AI benchmarks
#156Earlier quoted context omitted.
Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes. https://news.ycombinator.com/newsguidelines.html
What about the accusation that I was spreading false rumors? That's far more unkind
The guidelines apply to everyone. I didn't like that part of their comment either, and the comment would have been better without it, but they were put on the defensive by what had come before and they went on to explain their position. Your phrase "got caught with your hand in the cookie jar" is a swipe that the thread could also have done without. It's no big deal, and it applies to everyone on the subthread; we want everyone to avoid barbs like that on HN.
Re: Meta got caught gaming AI benchmarks
#157Earlier quoted context omitted.
I don't see anything in the article about being caught. Maybe I missed something?
[flagged]
Your comment would be better without the personal swipe in the first sentence.
Re: Meta got caught gaming AI benchmarks
#158Earlier quoted context omitted.
Unfortunately I wasn't able to get an interview with Netflix. > employees that all they needed to do is deliver what the company wanted How did this work out in practice and across teams? My experience at Meta within my team was that it would be almost impossible to determine what the company actually wanted from our team in a year. Goals kept changing and the existing incentive system works against this since other…
Netflix used to have a single level for engineers: Senior. That was the best decision they made, at least when they were small. It simply eliminated any incentive to work for promotion, and the result was magnificent: engineers would generally work on what's right, if they were ambitious. Or they could choose to finish what they were supposed to do. Netflix's culture deck explicitly said it was okay for employees to…
Imagine if sports teams worked this way. Hey you as a wide reciever didn't get promoted to QB - sorry we have to let you go!
Re: Meta got caught gaming AI benchmarks
#159I'm just shocked that the companies who stole all kinds of copyrighted material would again do something unethical to keep the bubble and gravy train going...
Yes, their worst fear is people figuring out that an AI chatbot is a strict librarian that spits out quotes but doesn't let you enter the library (the AI model itself). Because with 3D game-like UIs people can enter the library and see all their stolen personal photos (if they were ever online), all kind of monsters. It'll be all over YouTube. Imagine this but you remove the noise and can walk like in an art gallery…
Do you really think facebook's model weights contain all the facebook personal photos?
Re: Meta got caught gaming AI benchmarks
#160Last LLMs launches from top US AI labs were rather disappointing. Products seems under baked and mainly reactive... They oversold the capabilities of their future models when Deepseek landed to stay in the news at the time, but now that we look at the result it's not what everyone expected. And people are starting to question the hype, which is healthy but also very dangerous for these companies needing very large capital.
It seems that some labs like Meta are unto something, but it's more research material for now than a short term product. Interesting times.