Earlier quoted context omitted.
> with a limp > wheelchair You really took "cripple the model" to heart!
That wasn't my intention and I sincerely did not mean any offense.
Choosing an AI model: one prompt, 11 models, different results
81–90 of 105 posts
Re: Choosing an AI model: one prompt, 11 models, different results
#82Earlier quoted context omitted.
Doesn't most of this show up on google anyway?
Where do you think Google and co. source that data from? Having an actual, searchable, up to date menu is, just for a11y and accuracy far more valuable than someone’s poorly lit, low res upload of an photo taken from the menu circa 2019, just to name one advantage. Have found data from restaurants without their own webpages on Maps utterly inaccurate, even suggesting some that have been shut down for months to where…
Re: Choosing an AI model: one prompt, 11 models, different results
#83Re: Choosing an AI model: one prompt, 11 models, different results
#84Earlier quoted context omitted.
But... why do you want to look up a coffee shop online before going there? Honestly have never heard anyone say this in my life before
When I travel I have limited time, and if e.g. the coffee shop doesn't have the hallmarks of good coffee, I will save my time and find one that does
Re: Choosing an AI model: one prompt, 11 models, different results
#85Earlier quoted context omitted.
But... why do you want to look up a coffee shop online before going there? Honestly have never heard anyone say this in my life before
Why does anyone look up anything before going there? Maybe no one should have a website, everyone should go check everything out themselves in person like the telegraph hasn't been invented.
And why is it so wrong to go to a place to drink coffee without researching it in the first place?
Re: Choosing an AI model: one prompt, 11 models, different results
#86Re: Choosing an AI model: one prompt, 11 models, different results
#87Am I wrong or are these evaluations, while interesting, not really meaningful for anyone doing serious development work? I'm asking because I personally only use AI with specific and detailed instructions, building my projects piece-by-piece. I mostly don't look at the low level code and some of it I don't understand as much as I'd like, but I very much give much more technical instructions than a simple, two sentenc…
Seriously, I can't stand these tests of 1000 input tokens generating 1 million output tokens. You can't really convey anything interesting in 100 words, so like almost who cares what it produces? No one is providing value if the extent of their contribution is so small. Much more interesting is providing a million tokens of meaningful input and getting 1000 tokens out (high level critique of a detailed design doc, fi…
Re: Choosing an AI model: one prompt, 11 models, different results
#88Am I wrong or are these evaluations, while interesting, not really meaningful for anyone doing serious development work? I'm asking because I personally only use AI with specific and detailed instructions, building my projects piece-by-piece. I mostly don't look at the low level code and some of it I don't understand as much as I'd like, but I very much give much more technical instructions than a simple, two sentenc…
Seriously, I can't stand these tests of 1000 input tokens generating 1 million output tokens. You can't really convey anything interesting in 100 words, so like almost who cares what it produces? No one is providing value if the extent of their contribution is so small. Much more interesting is providing a million tokens of meaningful input and getting 1000 tokens out (high level critique of a detailed design doc, fi…
Re: Choosing an AI model: one prompt, 11 models, different results
#89Am I wrong or are these evaluations, while interesting, not really meaningful for anyone doing serious development work? I'm asking because I personally only use AI with specific and detailed instructions, building my projects piece-by-piece. I mostly don't look at the low level code and some of it I don't understand as much as I'd like, but I very much give much more technical instructions than a simple, two sentenc…
I'm repeatedly noticing that people working at big ai and tech companies are surprisingly not that... good... at using ai? It's like theyre doing a plausible thing to get something done and calling it a day
Re: Choosing an AI model: one prompt, 11 models, different results
#90I've been doing a lot of benchmarking for a long time now with a number of local models for the purposes of spam filtering. The major observation is that there is a lot of variance in model performance. This should not be surprising, as these are probabilistic machines based on random numbers, so your performance will vary from run to run. But this also means that any sort of evaluation of benchmark with a sample siz…