81–90 of 387 posts
I'm actually shocked that a company that has spent 25 years on finetuning search results for any random question people ask in the searchbox does not have a good, clean, dataset to train an LLM on. Maybe this is the time to get out the old Encyclopedia Britannica CD and use that for training input.
Why do people act like LLMs only hallucinate some of the time?
"Achieving the initial 80 percent is relatively straightforward since it involves approximating a large amount of human data, Marcus said, but the final 20 percent is extremely challenging. In fact, Marcus thinks that last 20 percent might be the hardest thing of all." 100% completely accurate is super-AI-complete. No human can meet that goal either. No, not even you, dear person reading this. You are wrong about som…
I grant you, my print Encyclopedia Britannica is not 100% accurate. But the difference between it and a LLM is not just a matter of degree: there's a "chain of custody" to information that just isn't there with a LLM.
Philosophers have a working definition of knowledge as being (at least†) "justified true belief."
Even if a LLM is right most of the time and yields "true belief", it's not justified belief and therefore cannot yield knowledge at all.
Knowledge is Google's raison d'etre and they have no business using it unless they can solve or work around this problem.
† Yes, I know about the Gettier problem, but is not relevant to the point I'm making here.
Something innocuous to a non ai scientist human but is otherwise fatal to the LLM data sets.
I'm actually shocked that a company that has spent 25 years on finetuning search results for any random question people ask in the searchbox does not have a good, clean, dataset to train an LLM on. Maybe this is the time to get out the old Encyclopedia Britannica CD and use that for training input.
It’s a bit weird since Google is taking over the “burden of proof”-like liability. Up until now, once user clicked on a search result, they mentally judged the website’s credibility, not Google’s. Now every user will judge whether data coming from Google is reliable or not, which is a big risk to take on, in my opinion.
Google did well in the old days for reasons. It beat alta vista and Yahoo by having better search results and a clean loading page. Since perhaps 08 (based on memory, that date might be off) or so, Google has dominated search, to the extent that it's not salient that search engines can be really questionable. Which is also to say, google dominated, people lost sight that searching and googling are different, that gives a lot of freedom for enshittification without people getting too upset or even quite realizing - it could be different and better
Earlier quoted context omitted.
They already know it’s a shit show. They are trying to push it along until it’s someone else’s fault.
I'm not convinced the executive layer is aware how dire the problem is. On one hand, their support for outsourcing programmes; "Training Indians on how to use AI", suggests they realize AI tooling without human cleanup is a crapshoot. On the other hand, they keep digging. This kind of gaslighting is an old and proven trick for genuinely rare problems , but it doesn't work if your issues are fairly common, as they'll…
I regularly speak to laypeople who assume that it's some magical thing without limits that makes their lives better. They are also 100% unaware of any applications that will actually make their lives better. End game occurs when those two disconnected thoughts connect and they become disinterested. The power users and engineers who were on it a year ago are either burned out or finding the limitations a problem as well now. There is only magical thinking, lies and hope left.
Granted there are some viable applications but they are rather less overstated than anything we have no and there are even negative side effects of those (think image classification, which even if it works properly, requires human review and there are psychological and competence things problems around that too).
Why do people act like LLMs only hallucinate some of the time?
The best trick the A.I. companies have pulled is getting us to refer to ‘bugs’ as ‘hallucinations.’ It sounds so much more sophisticated.
Why do people act like LLMs only hallucinate some of the time?
Earlier quoted context omitted.
> So Google hasn't used an LLM to generate and test weird queries ? What about simple manual testing? Seems to have skipped QA completely, automated or not.
There has been a lot of excitement recently about how using lower precision floats only slightly degrades LLM performance. I am wondering if Google took those results at face value to offer a low-cost mass-use transformer LLM, but didn’t test it since according to the benchmarks (lol) the lower precision shouldn’t matter very much. But there is a more general problem: Big Tech is high on their own supply when it come…