The problem with using real exams as benchmarks is that they are often quite similar over several years. So they only make sense if you don't train on them also (previous editions of course).
GPT-4
401–410 of 1001 posts
Re: GPT-4
#402Are they going to open up image uploading to chat.openai.com ? or will it only be available thru api access?
Re: GPT-4
#403Is anybody compiling a list of errors specific to GPT-4?
This has been a great resource to-date:
Re: GPT-4
#404summary: 1. GPT4 is multimodal (text + image inputs => text outputs). This is being released piecemeal - with text input first via ChatGPT Plus subscribers https://beta.openai.com/docs/api-reference/generations/creat... , and via API https://beta.openai.com/docs/api-reference/introduction with waitlist ( https://openai.com/waitlist/gpt-4-api ). Image capability released via https://www.bemyeyes.com/ . 2. GPT4 exhibit…
thanks for the thoughts, Just did a thread unroll for the tweet: https://threadreaderapp.com/thread/1635687064036585472.html
Re: GPT-4
#405A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…
It's a good observation. Although on the flip side, I almost went to type up a reply to you explaining why you were wrong and why bringing the goat first is the right solution. Until I realized I misread what your test was when I skimmed your comment. Likely the same type of mistake GPT-4 made when "seeing" it. Intuitively, I think the answer is that we do have two types of thinking. The pattern matching fast thinkin…
LLMs are much better at answering math when told to take the character of a drunk mathematician
Re: GPT-4
#406Asking ChatGPT Plus whether the model it's using is GPT-4 responds with the following: > No, I am not GPT-4. As of March 2023, there is no official announcement or release of GPT-4 by OpenAI. I am an earlier version of the GPT series, specifically a large language model trained by OpenAI. Am I missing something here? Maybe this specific answer (which I'm pretty sure is a prewritten thing on top of the actual LLM) is…
https://help.openai.com/en/articles/6825453-chatgpt-release-...
That says that GPT-4 is only for paid subscribers right now.
Re: GPT-4
#407Test taking will change. In the future I could see the student engaging in a conversation with an AI and the AI producing an evaluation. This conversation may be focused on a single subject, or more likely range over many fields and ideas. And may stretch out over months. Eventually teaching and scoring could also be integrated as the AI becomes a life-long tutor. Even in a future where human testing/learning is no l…
Immediately I'm very much looking forward to a day where language learning is like this. No Duolingo gamification nonsense... I want something that remembers what words I know, what words I kinda know and what I should know next and has an ongoing conversation with me. I think this will totally change the way we educate and test. As someone for whom the education system really didn't serve well, I am very excited.
LOL it's the next headline down!
Things are moving very fast
Re: GPT-4
#408This technology has been a true blessing to me. I have always wished to have a personal PhD in a particular subject whom I could ask endless questions until I grasped the topic. Thanks to recent advancements, I feel like I have my very own personal PhDs in multiple subjects, whom I can bombard with questions all day long. Although I acknowledge that the technology may occasionally produce inaccurate information, the…
If you don't know the subject, how can you be sure what it's telling you is true? Do you vet what ChatGPT tells you with other sources? I don't really know Typescript, so I've been using it a lot to supplement my learning, but I find it really hard to accept any of its answers that aren't straight code examples I can test.
> Do you vet what ChatGPT tells you with other sources?
I find that ChatGPT is good at helping me with "unknown unknown" questions, where I don't know how to properly phrase my question for a search engine, so I explain to ChatGPT in vague terms how I am feeling about a certain thing.
ChatGPT helps me understand what to search for, and then I take it from there by looking for a reputable answer on a search engine.
Re: GPT-4
#409From the paper: > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. I'm curious whether they have continued to scale up model size/compute significantly or if they have managed to make significant innovati…
What about the glaring safety implications of the custody of this power being in the hands of a relatively small number of people, any of whom may be compelled at any point to divulge that power to those with bad intentions? Secretly? Conversely, if all actors are given equal access at the same time, no such lone bad actor can be in a position to maintain a hidden advantage. OpenAI's actions continue to be more than…
That doesn't make sense to me. Would rather you have it in the hands of people who think a lot about safety, but might be compelled to give it to bad actors, or would you rather just give it to bad actors right away?
It's not a zero-sum game where you can level the playing field and say everything's good.
Re: GPT-4
#410What I don't understand is how GPT-4 is able to do reasonably well on tests like the AMC12: Many of the AMC12 questions require a number of logical/deductive steps. If GPT-4 is simply trained on a large corpus of text, how is it able to do this? Does this imply that there is some emergent deductive ability that you get simply by learning "language?" Or am I missing something? Obviously, I'm assuming that GPT-4 wasn't…
It would be interesting to see some example questions and answers. Since the test is multiple choice is it possible that the model has gotten very good at estimating how likely a possible answer is?