701–710 of 1001 posts
https://www.nerdwallet.com/article/investing/bullish-vs-bear...
The current discussion about LLMs guarantees that both positive and negative expectations are a valid title for an article xD
People who don't work in tech have no idea how hard it is to do certain things at scale. Skilled tech people are severely underappreciated. From a sub-tweet: >> no LLM should ever output a url that gives a 404 error. How hard can it be? As a developer, I'm just imagining a server having to call up all the URLs to check that they still exist (and the extra costs/latency incurred there)... And if any URLs are missing,…
Very easy. We can replace the error handler with a bullshit generator, and these people will be satisfied, as the whole idea is bullshit by the way.
Earlier quoted context omitted.
I want to sit next to you and stop you every time you use your LLM and say, “Let me just carefully check this output.” I bet you wouldn’t like that. But when I want to do high quality work, I MUST take that time and carefully review and test. What I am seeing is fanboys who offer me examples of things working well that fail any close scrutiny— with the occasional example that comes out actually working well. I agree…
What's also scary is that we know LLMs do fail, but nobody (even the people who wrote the LLM) can tell you how often it will fail at any particular task. Not even an order of magnitude. Will it fail 0.2%, 2%, or 20% of the time? Nobody knows! A computer that will randomly produce an incorrect result to my calculation is useless to me because now I have to separately validate the correctness of every result. If I nee…
Benchmarks could track that too - I don't know if they do, but that information should actually be available and easy to get.
When models are scored on e.g. "pass10", i.e. pass the challenge in under 10 attempts, and then the benchmark is rerun periodically, that literally produces the information you're asking for: how frequently a given model fails at particular task.
> A computer that will randomly produce an incorrect result to my calculation is useless to me because now I have to separately validate the correctness of every result.
For many tasks, validating a solution is order of magnitudes easier and cheaper than finding the solution in the first place. For those tasks, LLMs are very useful.
> If I need to ask an LLM to explain to me some fact, how do I know if this time it's hallucinating? There is no "LLM just guessed" flag in the output. It might seem to people to be "miraculous" that it will summarize a random scientific paper down to 5 bullet points, but how do you know if it's output is correct? No LLM proponent seems to want to answer this question.
How can you be sure whether a human you're asking isn't hallucinating/guessing the answer, or straight up bullshitting you? Apply the same approach to LLMs as you apply to navigating this problem with humans - for example, don't ask it to solve high-consequence problems in areas where you can't evaluate proposed solutions quickly.
Earlier quoted context omitted.
If it works for you... I use LLMs. They're somewhat useful if you're on a non niche problem. They're also useful instead of search engines, but that's because search has been entshittified more than because a LLM is better. However 90% of the marketing material about them is simply disgusting. The bigwigs sound like they're spreading a new religion, and most enthusiasts sound like they're new converts to some sect. I…
> I use LLMs. They're somewhat useful if you're on a non niche problem. They're also useful instead of search engines... The problem for me is that I could use that type of assistance precisely when I hit that "niche problem" zone. Non-niche problems are usually already solved. Like search. Popular search engines like Google and Bing are mostly garbage because they keep trying to shove gen AI in my face with made up…
Tough luck. On the other hand, we're still justified in asking for money to do the niche problems with our fleshy brains, right? In spite of the likes of Altman saying every week that we'll be obsoleted in 5 years by his products. Like ... cold fusion? Always 5 years away?
[I have more hope for cold fusion than these "AIs" though.]
> Popular search engines like Google and Bing are mostly garbage because they keep trying to shove gen AI in my face with made up answers.
No they became garbage significantly before "AI". Google at least has gradually reduced the number of results returned and expanded the search scope to the point that you want a reminder of the i2c api syntax on a raspberry pi and they return 20 beginner tutorial results that show you how to unpack the damn thing and do the first login instead.
Earlier quoted context omitted.
At non-extremely niche tasks they fail as well. I should start collecting examples, if only for threads like this. Recently I tried to llm a tsserver plugin that treats lines ending with "//del" as empty. You can only imagine all the sneaky failures in the chat and the total uselessness of these results. Anything that is not literally millions (billions?) of times in the training set is doomed to be fantasized about…
LLMs are basically a search engine for Stack Overflow and Github that doesn't suck as bad as Google does. If your job is copy-pasting from Stack Overflow then LLMs are an upgrade.
I genuinely don't understand why some people are still using X.
I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…
More like we note the frequency with which these tools produce shallow bordering on useless responses, note the frequency with which they produce outright bullshit, and conclude their output should not be taken seriously. This smells like the fervor around ELIZA, but with several multinational marketing campaigns behind it pushing.
I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…
I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…
I think some people just aren't using them correctly or don't understand their limitations.
They are especially helpful for helping me get over thought paralysis when starting new project.
I get so confused on this. I play around, test, and mess with LLMs all the time and they are miraculous. Just amazing, doing things we dreamed about for decades. I mean, I can ask for obscure things with subtle nuance where I misspell words and mess up my question and it figures it out. It talks to me like a person. It generates really cool images. It helps me write code. And just tons of other stuff that astounds me…
Look man, and I'm saying this not to you but to everyone who is in this boat; you've got to understand that after a while, the novelty wears off. We get it. It's miraculous that some gigabytes of matrices can possibly interpret and generate text, images, and sound. It's fascinating, it really is. Sometimes, it's borderline terrifying. But, if you spend too much time fawning over how impressive these things are, you m…
They are useful enough that they can passably replace (much more expensive) humans in a lot of noncritical jobs, thus being a tangible tool for securing enterprise bottom lines.