After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
The power openai will hold above everyone else is just too much. They will not allow their AI as a service without data collection. That will be a big pill to swallow for the EU.
OpenAI have been consistently ahead of everyone but the others are not far behind. Everyone is seeing the dollar signs, so I'm sure all big players are dedicating massive resources to create their own models.
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
Reading the press release, my jaw dropped when I saw 32k. The workaround using a vector database and embeddings will soon be obsolete.
Cost is still a concern, so workarounds to reduce context size are still needed
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
Do you think this will be enough context to allow the model to generate novel-length, coherent stories? I expect you could summarize the preceding, already generated story within that context, and then just prompt for the next chapter, until you reach a desired length. Just speculating here. The one thing I truly cannot wait for is LLM's reaching the ability to generate (prose) books.
I've seen that it can also generate 25k words. That's about 30-40% of the average novel
As the world marvels at the astonishing capabilities of OpenAI's GPT-4, I find myself contemplating the rapid acceleration of AI and machine learning, and the evolutionary impact it is having on our lives. Naturally, I turned to GPT-4 to assist me in these thoughts. GPT-4's human-level performance on professional and academic benchmarks - such as the 88th percentile on the LSAT and the 89th on SAT Math - is a testame…
A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…
I gave it a different kind of puzzle, again with a twist (no solution), and it spit out nonsense. "I have two jars, one that can hold 5 liters, and one that can hold 10 liters. How can I measure 3 liters?" It gave 5 steps, some of which made sense but of course didn't solve the problem. But at the end it cheerily said "Now you have successfully measured 3 liters of water using the two jars!"
That's a good example which illustrates that GPT (regardless of the number) doesn't even try to solve problems and provide answers, because it's not optimized to solve problems and provide answers - it is optimized to generate plausible text of the type that might plausibly be put on the internet. In this "genre of literature", pretty much every puzzle does have a solution, perhaps a surprising one - even those which are logically impossible tend to have actual solutions based on some out-of-box thinking or a paradox; so it generates the closest thing it can, with a deus ex machina solution of magically getting the right answer, since probably even that is more likely as an internet forum answer as proving that it can't be done. It mimics people writing stuff on the internet, so being wrong or making logic errors or confidently writing bullshit or intentionally writing lies all is plausible and more common than simply admitting that you have no idea - because when people have no idea, they simply don't write a post about that on some blog (so those situations don't appear in GPT training), but when people think they know, they write it up in detail in a confident, persuasive tone even if they're completely wrong - and that does get taught to GPT as an example of good, desirable output.
From the livestream video, the tax part was incredibly impressive. After ingesting the entire tax code and a specific set of facts for a family and then calculating their taxes for them, it then was able to turn that all into a rhyming poem. Mind blown. Here it is in its entirety: --- In the year of twenty-eighteen, Alice and Bob, a married team, Their income combined reached new heights, As they worked hard day and…
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
I must have missed the part when it started doing anything algorithmically. I thought it’s applied statistics, with all the consequences of that. Still a great achievement and super useful tool, but AGI claims really seem exaggerated.
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
Do you think this will be enough context to allow the model to generate novel-length, coherent stories? I expect you could summarize the preceding, already generated story within that context, and then just prompt for the next chapter, until you reach a desired length. Just speculating here. The one thing I truly cannot wait for is LLM's reaching the ability to generate (prose) books.
E.g. Kafka's metamorphosis fits entirely in the context window I believe, so short novellas might be possible. But I think you'd still definitely need to guide GPT4 along, I imagine without for example a plan for the plot formulated in advance, the overarching structure might suffer a lot / be incoherent.
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
Reading the press release, my jaw dropped when I saw 32k. The workaround using a vector database and embeddings will soon be obsolete.
That’s like saying we’ll not need hard drives now that you can get bigger sticks of RAM.