101–110 of 124 posts
Earlier quoted context omitted.
Except for things they happen to know something about.
Unfortunately, too few people are making the obvious leap from "LLMs aren't great for topics I have expertise in" to "maybe that means LLMs aren't actually great for the other topics either."
Earlier quoted context omitted.
Yesterday my wife burst into my office: "You used AI to generate that (podcast) episode summary, we don't sound like that!" In point of fact, I had not. After the security reporting issue, the next problem on the list is "trust in other people's writing".
Already a big problem in art, people go on witch hunt over what they think are signs of AI use. It's sad because people that are ok with AI art are still enjoying the human art just the same. Somehow their visceral hate of AI-art managed to ruin human art for themselves as well.
But instead we had a 'non-profit' called 'Open'AI that irresponsibly unleashed this technology on the world and lied about its capabilities with no care of how it would affect the average person.
The solution isn't to block aggressively or to allow everything, but to prioritize. Put accounts older than the AI boom at the top, and allow them to give "referrals", ie stake a part of their own credibility to boost another account on the priority ladder. Referral systems are very efficient at filtering noise.
There is also the possibility that in trying to get someone to refer me I give enough details that the trusted person can submit instead of me and claim credit.
> The model has no concept of truth—only of plausibility. This is such an important problem to solve, and it feels soluble. Perhaps a layer with heavily biased weights, trained on carefully curated definitional data. If we could train in a sense of truth - even a small one - many of the hallucinatory patterns disappear. Hats off to the curl maintainers. You are the xkcd jenga block at the base.
The "fact database" is the old AI solution, e.g. Cycorp; it doesn't quite work either. Knowing what is true is a really hard, unsolved problem in philosophy, see e.g. https://en.wikipedia.org/wiki/Gettier_problem . The secret to modern AI is just to skip that and replace unsolvable epistemology with "LGTM", then sell it to investors.
- dictionary definitions - stable apis for specific versions of software - mathematical proofs - anything else that is true by definition rather than evidence-based
(i realize that some of these are not actually as stable over time as they might seem, but they ought to do good enough with the pace that we train new models at).
If you even just had an MOE component whose only job was verifying validity against this dataset in chain-of-thought I bet you'd get some mileage out of it.
Even more so when there is a bounty payout.
Refundable if the PR/report is accepted.
> This is the fundamental problem: AI can generate the form of security research without the substance. I think this is the fundamental problem of LLMs in general. Some of the time looks just enough right to seem legitimate. Luckily the rest of the time it doesn’t.
A parallel to AI-slop has existed for generations now out here in meatspace: Administrative/legal people on the periphery of a technical field (though possibly alas, at the top of the org's command chain) who do not at all understand what technical terms signify, but having seen hundreds of sentences produced by real experts, become able to themselves string together plausible-looking assertions.
Any large enough organization gathers them en mass to cloud real development work with "compliance."
Earlier quoted context omitted.
Except for things they happen to know something about.
Unfortunately, too few people are making the obvious leap from "LLMs aren't great for topics I have expertise in" to "maybe that means LLMs aren't actually great for the other topics either."
> This is the fundamental problem: AI can generate the form of security research without the substance. I think this is the fundamental problem of LLMs in general. Some of the time looks just enough right to seem legitimate. Luckily the rest of the time it doesn’t.
Just add a country IP ban, we all know who is submitting these reports. Remember Hacktoberfest?
That's a game of whack-a-mole, they'd just use a VPN. And besides, no we don't "all know" who is submitting these reports, that's a generalization.
Just add a country IP ban, we all know who is submitting these reports. Remember Hacktoberfest?
> Just add a country IP ban, we all know who is submitting these reports. As much as I'd like to see Russia, China and India disconnected off of the wide Internet until they clean up shop with abusive actors, the Hacktoberfest stuff you're likely referring to doesn't have anything to do with your implication - that was just a chance at a free t-shirt [1] that caused all the noise. In ye olde times, you'd need to take…