If it's not already obvious, LLMs are going to be doing most of the mental work currently performed by doctors, lawyers, accountants, etc. I have already nearly stopped using Google search for anything, in favor of GPT-4. GPT-4 has helped me very quickly prototype things that I normally would have had to spend hours researching. GPT-4 has also created custom curriculum for me to help me learn various things for which…
I’m very glad you set the argument this way, since it very neatly sets the stage for the core disagreement to be argued. > Simply very good information synthesizers > Simple very good syntactic synthesizers I argue that theorists tend to adhere to the former, and practitioners the latter. Consider prototyping: For an expert, it costs nothing (or near nothing) to check the output. You even start better, because you kn…
I think it is clear that GPT-4 contains a LOT of information. You can ask it explicit factual questions and it often gets the right answer. When it gets the answer wrong, its wording is typically syntactically correct English, or syntactically correct code.
I'd argue that just because some of the errors it commits are "semantic errors" such as calling a method by an incorrect (but often similar) name in a segment of code or printing false statement in well-crafted prose, it nonetheless gets a lot of the semantics right.
What is reasoning besides a set of language patterns that we define as valid reasoning? Imagine evaluating statements in various formal logics. Nonsense in one can be valid in another based on semantic rules alone.
One could derive the model (model as in model-theoretic semantics) of a formal logical system by sampling a list of valid and invalid statements.
LLMs are doing that kind of thing, it seems. There are gaps, but they are not necessarily gaps that the LLM itself cannot notice.
For example, I will often ask GPT-4 to formulate a plan for something or to create a list of priorities/considerations for an undertaking. I will then ask it to draft an initial plan. After that I will ask it to review/critique its draft based on the initial goals. It typically points out exactly the kinds of gaps that a human would point to as deficiencies that indicate the LLM is not reasoning.
In my view, this indicates that the "knowledge" of how to do the task was always aviailable to the LLM, but the interface (or some aspect of the internal implementation) did not allow the knowledge to be applied all at once. This is not necessarily dissimilar from human intellectual work, in which drafts and self-critiquing is not an unreasonable series of steps.
I have done some work with parallel promopts and various "roles" for different LLM interlocutors toward the same task. While it does sometimes go off the rails, it seems clear that multiple prompts with role-based instructions do achieve a greater level of analytical rigor than a single prompt.