Live data from Hacker News

Scientists should use AI as a tool, not an oracle

aisnakeoil.com

101–110 of 113 posts

Re: Scientists should use AI as a tool, not an oracle

#101
post #22

I feel like 90% of AI discussions online these days can be shut down with “a probabilistic syllable generator is not intelligence”

Even people who _know_ that often seem to have difficulty intuitively believing it, is the trouble; it's very good at _appearing_ to be intelligent, good enough that even people who should know better sometimes think that the correctness problems are just a case of "need more GPUs", rather than insoluble.

Re: Scientists should use AI as a tool, not an oracle

#102
post #39

Earlier quoted context omitted.

Wait, no, it's "incorrect" in the sense that you asked it to do something, and the thing it gives you doesn't accomplish the task. I asked it "what is the PS3 game where the full version of To Kill a Mockingbird is in there?" and it responded back with "The Sabateour", when the correct answer would have been "The Darkness". That is incorrect by most definitions of the word, whether or not it's a consequence of the tr…

You're anthropomorphising too much. The machine did the correct thing; it's just that it is a prediction machine, not a magic question answering machine, so its 'correct' may not be what you wanted. > I suppose we could get into details about epistemology and ontology about the nature of what an answer "is" The machine has no concept of an 'answer'; when people call these things autocomplete on steroids, they're not…

It really isn’t anthropomorphizing at all. If I treat it like a black box it really is quite simple. I gave it an input in the form of English text, there is an objectively correct and incorrect form of English text that responds to the input, it gave me the incorrect one.

Literally no one here disputes that the it’s a glorified autocomplete, but that is completely irrelevant to if it correctly answered a question.

I find this kind of pedantry extremely annoying because it sounds insightful without actually saying anything. Like, no shit, it’s just doing what its algorithm dictates, no one, and I mean no one disputed that. The question of correctness and incorrectness falls into “how accurately did it respond to my query?”

Re: Scientists should use AI as a tool, not an oracle

#103

The worst is having random people questioning your expertise because of what ChatGPT told them.

Doctors had this moment when Google first came out

To be fair, I came across doctors who are no better than a static webpage from the CDC. I fire those doctors pretty quickly.

Re: Scientists should use AI as a tool, not an oracle

#106

Maybe they should also call it "curve fitting" instead of "AI" so they don't need to call a "poor fit" a "hallucination"

It's all very simple, eh?

Look, it's a bit late here so I don't really have time to fully refute your sophisticated argument. But let me ask you this: if AI is not curve fitting, what is it then?

Re: Scientists should use AI as a tool, not an oracle

#108

Earlier quoted context omitted.

Sure, if you also assume people follow whatever advice so given. They won't, even before the covert influence effort becomes popular knowledge, as it inevitably will. This destroys consumer trust in your product after you have successfully made that product indispensable, thus opening up a previously impossible vacuum in epistemology and thus access to power.

For the record I believe it to be immoral to manipulate humanity in this way. And I also believe it might be bad for bussiness. I was just trying to explain to the guy above what I think the guy above that meant.

'The guy above' is also me :) And yeah, I get it. I guess the thing I'm trying to get at by extension and example is just how hard a problem this is, and maybe also that the formulation given assumes LLMs have a level of control over human behavior that not only doesn't exist but in the general case sort of can't since the LLM's user is always free not to take its advice. (At the very least, if humans have no option but to follow LLM instructions, something has gone much more badly wrong than the risk of there being poor instructions...)

In general, I think it's a good example of the kind of social problem tech can make a lot worse but no better: when a society has lost its grasp of epistemology, multiplying the amount of information available, at a net decrease in quality and reliability, merely multiplies the scope of the problem.

Re: Scientists should use AI as a tool, not an oracle

#110

Earlier quoted context omitted.

It's all very simple, eh?

Look, it's a bit late here so I don't really have time to fully refute your sophisticated argument. But let me ask you this: if AI is not curve fitting, what is it then?

I'm objecting to the implication that if we start referring to it as curve fitting rather than AI, then thinking about it becomes easier and it becomes less likely that we will make a huge collective mistake in thinking about it.

I'm not saying there aren't a few possible mistakes that do become less likely if we switch to "curve fitting, but I suspect that it does not matter much either way on the most serious mistakes.

Post reply on HN