Cargo Cult AI

How do you know it's actually conducting the proofs and not that the proofs and related data were already part of the dataset?

Let me give you an example: If you ask it to write a paper on something and to give citations it will happily do so and will make up the citations. It will link you to websites that don't exist, or papers that are completely wrong. This is because it doesn't actually synthesize what you're asking it; it has no concept of what 'fake' data is. It simply does what you ask. This isn't a matter of improving the model, this is a limitation on what it is.

Ignoring what anthropologists might say about our use of "cargo cult" as a derogatory term today, you make me wonder what an AI cargo cult should look like instead.

Is it setting up a Turing Test chamber and leaving a seat open for the AI?

Or trying to breed industrial robots?

Or is it downloading someone else's model and trying to use it as a magic box without any concern for how it was trained nor any real intent to validate its output on your newly chosen domain?

You can give GPT-4 some data and ask it to make a hypothesis to explain it, give it tools to test the hypothesis, and have it try to establish causation. You can literally do that now quite easily with ChatGPT abilities like plugins and it will prove the premise of the article false.

An interesting contrast can be drawn between this article and a criticism like Yann Lecun's. In that his actually has substance. https://youtu.be/DokLw1tILlw Although he is also wrong about the capabilities of LLMs.

Certainly LLMs are not the end of AI research. They have various types of deficiencies and some missing capabilities that humans have. And are not alive.

But GPT-4 can definitely complete scientific experiments.

I couldn't help myself but I think at least GPT-4 gets you:

USER What is principles can be extracted from this saying:

>Is the ability to think scientifically the defining essence of intelligence? ; Physicist Carl Sagan once wrote that "science is more than a body of knowledge; it is a way of thinking." This type of thinking requires skeptical rigor and brutal honesty to thoroughly investigate,... I think the key to brutal honesty is the ability to deliver and accept brutal honesty between peers and rivals. This is the skill you f'ing idiots seem to be losing, because it hinders hearing the autistic perspectives which are quite useful in science. I dropped the completely meaningless f-bomb not to insult anybody but to test your ability to read the sentence without the most basic of intensifiers, what I like to call the f-italics. https://www.mit.edu/~jcb/tact.html

ASSISTANT 1. The importance of scientific thinking: The saying emphasizes the value of adopting a scientific mindset in understanding and problem-solving.

2. Balance of skepticism and openness: A scientific thinker should be rigorous and skeptical in their analysis but also open to new insights, even when these challenge previously held opinions.

3. Brutal honesty: To engage in a productive scientific discourse, it is essential to be brutally honest, both in terms of providing feedback to others and in accepting any criticism directed at one's own work or ideas.

4. Valuing diversity of perspectives: It is important to consider and respect diverse viewpoints in a scientific discussion, including those that deviate from the norm or that challenge mainstream thinking.

5. Emotional resilience: To thrive in a rigorous and honest intellectual environment, one must develop the ability to remain focused on the message and not be derailed by strong language or perceived slights.

6. Effective communication: Being able to deliver honest feedback or critique in a way that is both clear and constructive is a vital skill, even if it involves using syntactic intensifiers, like the f-bomb, to emphasize a point.

You don't, at this point. You also don't know whether a person is lying to you or not. Or whether the person believes and replicates some utter nonsense. Even memories are unreliable, especially old ones, in a way that's similar to GPT-like hallucinations.

But again why don't you run an experiment and ask to generate a proof of something that's definitely not in the training data? Like try something scientific instead of abstract philosophy?