91–100 of 119 posts
Earlier quoted context omitted.
LOL. Providing a counter example is far from just an anecdote.
An anecdote is an anecdote is an anecdote
The claim was this is not possible. I show it happening. That's a proof.
You're confused because the example is my own project. But I'm not merely describing it - I am giving you full access to it to confirm or reject my argument.
Not anecdotal. Saying it three times does not make it true.
Sycophancy is the "stickiness factor" of AI analogous to that of Social Media.
For some background read Jagged Intelligence: The Dangerous Unknowns at the Heart of LLMs - https://news.ycombinator.com/item?id=48577159
Here is an experiment that i did;
Ask AI to build a "Character Profile"(in the broadest sense) of a person based on their available public writings. This requires "commonsense reasoning" (https://en.wikipedia.org/wiki/Commonsense_reasoning), understanding human motivations and behaviour, context, assumptions, societal knowledge etc.
I know "me" and so i asked AI to use my HN comments/submissions as input :-) The sycophancy/flattery/praise was quiet excessive. I am realistic and old enough to not need ego-soothing (a little is fine but a lot makes me suspicious) and so i asked AI whether these phrases were not too over-the-top. It apologized and agreed to drop the fluff. I then asked it to identify job roles (any) for which i might not be a good fit given my character profile. This acts as an external constraint which focuses attention on shortcomings and hence forces the AI to look at the other side of the coin. Now the results were better and more in line with what some Human may deduce from my HN persona (which obviously is not my complete real-life persona) but still wanting in many aspects.
The above is a perfect example of "Jagged Intelligence" exhibited by AI. Excellent in formal symbolic manipulation sciences, regurgitation and simple reasoning but highly deficient in human-like commonsense reasoning.
Earlier quoted context omitted.
Perhaps only a small minority want quality out of information product, while most want satisfying simplicity. This is definitely how the news plays out. I recently built an ai chart product that attempts to add more nuance by injecting extra models to jump in when the first model misses the mark. ( http://pellmell.au So far it seems like only a minority of people like it. The main feedback is that answers are confusi…
I learnt a long time ago that most people don't want their problems solved, they just want to vent.
"They want to vent" :-
"they want to be heard" :-
"they want to be understood" :-
"they want to know they are understood" :-
"they want to feel they matter to and are safe with the people they value"
Earlier quoted context omitted.
> The problem is that the crackpots are out there writing, and this is snarfed into the training data. No, the problem is the chatbot pretends to be an AI answer engine. Recognise it is simply a zero-intelligence search engine, and you'll not be surprised at all to get crackpot writings from the web. > write in a way that reintroduces rigidity ... and do not be surpised to get rigid crackpot writings from the web.
> Recognise it is simply a zero-intelligence search engine I think that this highlights the problem: it's not even a proper search engine. If it were, it would find existing material and link to it. The chatbot instead re-elaborates the message. This is different enough to have legal consequences (see the recent German ruling). By squashing everything together, one loses the ability to evaluate sources, including the…
No, you wouldn't because you're a fucking machine.
I have an alternative view which is that sycophancy erodes trust in AI, from those who are not seeking validation from a machine, but information or advice. If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a…
Earlier quoted context omitted.
An anecdote is an anecdote is an anecdote
A proof isn't an anecdotal story. The claim was this is not possible. I show it happening. That's a proof. You're confused because the example is my own project. But I'm not merely describing it - I am giving you full access to it to confirm or reject my argument. Not anecdotal. Saying it three times does not make it true.
If you want to have a whack at a hard problem, have a try at the cryptographic puzzle in Noita if you want to see the silly failure modes of these systems. Lots of people have tried to solve it with AI and nothing's budged. If you can solve it, great.
I have an alternative view which is that sycophancy erodes trust in AI, from those who are not seeking validation from a machine, but information or advice. If you're looking for advice in a situation where you are not sure of what the correct choices are, you will find LLM chat AI to go in circles. It says one thing. Something is dodgy about it, so you raise a tentative objection (as a non-expert). The thing does a…
One key point is that - “pick any topic where people disagree” - most knowledge-seeking I perform via a chat bot is to get a sense of what most people generally agree upon. A quick poll of the zeitgeist of the corpus it was trained on. What used to take me 10-15 minutes on Google now takes me 2-3 minutes with Gemini.
That holds true for Claude code too - I’m relying upon the fact that other people have already solved my problem, or have at least solved the components I need to tie together into a solution for my problem - I’m just fast forwarding through the process of digging them up myself and making sure they’re interoperable. The lion’s share of the tasks I set forth for an LLM is just ‘go find me something like this.’
How some people get from that, to weeks of AI-triggered mania, I simply do not understand.
Earlier quoted context omitted.
A proof isn't an anecdotal story. The claim was this is not possible. I show it happening. That's a proof. You're confused because the example is my own project. But I'm not merely describing it - I am giving you full access to it to confirm or reject my argument. Not anecdotal. Saying it three times does not make it true.
What things have you tried that didn't go well? At this point, that's more interesting. If you want to have a whack at a hard problem, have a try at the cryptographic puzzle in Noita if you want to see the silly failure modes of these systems. Lots of people have tried to solve it with AI and nothing's budged. If you can solve it, great.
What does your new subject have to do with the fact that the previous arguments were all pretty much baloney?
Pointing to a problem that has not been solved is completely unrelated to the miraculous things that have been solved.
And I thought you'd decided (three times now) that I'm not worth talking to?
Earlier quoted context omitted.
What things have you tried that didn't go well? At this point, that's more interesting. If you want to have a whack at a hard problem, have a try at the cryptographic puzzle in Noita if you want to see the silly failure modes of these systems. Lots of people have tried to solve it with AI and nothing's budged. If you can solve it, great.
Why are you changing the subject? What does your new subject have to do with the fact that the previous arguments were all pretty much baloney? Pointing to a problem that has not been solved is completely unrelated to the miraculous things that have been solved. And I thought you'd decided (three times now) that I'm not worth talking to?