371–380 of 1001 posts
I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…
A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…
How close are we to handing this thing a desktop and an internet connection with the prompt "ok now make gpt-5"? In fact, the models appear to be already kind of doing that? With the fuzzy layer of the humans still in the loop.
It'd be interesting to test this with gpt 4.
A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…
LLMs aren’t reasoning about the puzzle. They’re predicting the most likely text to print out, based on the input and the model/training data. If the solution is logical but unlikely (i.e. unseen in the training set and not mapped to an existing puzzle), then the probability of the puzzle answer appearing is very low.
What I don't understand is how GPT-4 is able to do reasonably well on tests like the AMC12: Many of the AMC12 questions require a number of logical/deductive steps. If GPT-4 is simply trained on a large corpus of text, how is it able to do this? Does this imply that there is some emergent deductive ability that you get simply by learning "language?" Or am I missing something? Obviously, I'm assuming that GPT-4 wasn't…
GPT4:
In the quest for AGI's creation,
Debates swirled in a whirlwind gyration,
But this limerick's plight,
Won't settle the fight,
For the answer's still lost in translation.
I think it's interesting that they've benchmarked it against an array of standardized tests. Seems like LLMs would be particularly well suited to this kind of test by virtue of it being simple prompt:response, but I have to say...those results are terrifying. Especially when considering the rate of improvement. bottom 10% to top 10% of LSAT in What are the implications for society when general thinking, reading, and…
There's a sample of GPT-4 acting as a "Socratic tutor" teaching a student how to solve a high school math problem. If that sample is representative, it means GPT-4 has a theory of other people's minds. Or it is so good at emulating one that it doesn't matter? I'm not sure where the "stochastic parrot" argument goes now.
Seems like OpenAI is forecasting massive changes to the job market. I highly recommend reading page 18 of the research paper. "GPT-4 or subsequent models may lead to the automation of certain jobs.[81] This could result in workforce displacement.[82] Over time, we expect GPT-4 to impact even jobs that have historically required years of experience and education, such as legal services.[83]"
I work at company that uses AI to automate about ⅓ of the job of trained licensed professionals. Looking at GPT4 those licensed professionals are now completely irrelevant. It's going to take years to build the supporting software around gpt4 to completely eliminate those jobs, but today I am convinced that we are on the verge of massive unemployment. Today thousands of job types have just been made redundant. What s…
The lack of progress here, says a lot more about is NOT happening as an AI paradigm change. Still a glorified pattern matching and pattern creation engine, even if a very impressive one.