Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4

openai.com

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: GPT-4

#841

Post by jchanimal »

graboid wrote: post #648
hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

Do you think this will be enough context to allow the model to generate novel-length, coherent stories? I expect you could summarize the preceding, already generated story within that context, and then just prompt for the next chapter, until you reach a desired length. Just speculating here. The one thing I truly cannot wait for is LLM's reaching the ability to generate (prose) books.

It wasn’t that hard to work in chunks and write a book on GPT-3, can only be easier. https://docs.google.com/document/d/1vx6B6WuPDJ5Oa6nTewKmzeJM...

Re: GPT-4

#842

Post by Madmallard »

ml_basics wrote: post #136

From the paper: > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. I'm curious whether they have continued to scale up model size/compute significantly or if they have managed to make significant innovati…

Open AI more like Closed AI

Safety has nothing to do with it. It's an easy tack on for them because of popular fear of AGI.

It's all about power over the market.

Cringe.

Re: GPT-4

#843

Post by hnfong »

swatcoder wrote: post #590
mk_stjames wrote: post #577

I just finished reading the 'paper' and I'm astonished that they aren't even publishing the # of parameters or even a vague outline of the architecture changes. It feels like such a slap in the face to all the academic AI researchers that their work is built off over the years, to just say 'yeah we're not telling you how any of this is possible because reasons'. Not even the damned parameter count. Christ.

In the old days of flashy tech conferences, that was precisely the sign of business-driven demo wizardry. The prerecorded videos, the staff-presented demos, the empty hardware chassis, the suggestive technical details, etc They have “reasons” for not giving away details, but there are good odds that the ultimate reason is that this is a superficial product update with a lot of flashy patchwork rather than that fundam…

No, the reason is they don’t want other companies to replicate their results so that they can maintain their first mover advantage.

You can use the product today, right now.

Re: GPT-4

#844

Post by fastball »

SergeAx wrote: post #793
hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

Please. Language model cannot "reason", it can just show next most probable word based on text corpus downloaded from the internet.

What do you mean by "next most probable word"? How do you calculate the probabilities of words appearing in a sentence that has never actually existed?

Re: GPT-4

#845

Post by hnfong »

lolsal wrote: post #578
maxdoop wrote: post #265

The comments on this thread are proof of the AI effect: People will continually push the goal posts back as progress occurs. “Meh, it’s just a fancy word predictor. It’s not actually useful.” “Boring, it’s just memorizing answers. And it scored in the lowest percentile anyways”. “Sure, it’s in the top percentile now but honestly are those tests that hard? Besides, it can’t do anything with images.” “Ok, it takes imag…

I’m one of these skeptics, but it’s not moving the goalposts. These goalposts are already there, in some sort of serial order that we expect them to be reached. It is good that when tech like this satisfied one of the easier/earlier goalposts, that skeptics refine our criticism based on evidence. You will see skepticism until it is ubiquitous; for example, Tesla tech - it’s iterative and there are still skeptics abou…

It’s one thing to be skeptical of the state of art and only believe something when you actually see it working (a useful antidote against vapor ware)

It’s another to keep making wrong assertions and predictions about the pace of advancement because of a quasi-religious belief that humans with meat-brains are somehow fundamentally superior .

Re: GPT-4

#846

Post by antondd »

techfoodie123 wrote: post #774

serious question for everyone: what are you planning to do when these LLMs replace our jobs? it seems it won't be long before a handful of tech employees will be all even the largest of companies will need, and maybe a few years after that the role will have changed so much there's no need for a single dedicated tech employee. i am terrified i imagine i should shift to some physical work. carpentry, real estate... so…

Assuming some form of UBI is implemented and AI replaces most tech/service-related jobs, there will still be plenty of work for all of us to do. In no particular order: cleaning our environment, planting new trees, removing trash from oceans, engaging in archaeology, conducting research, providing homes for animals, rebuilding war-torn countries, demining land, and so on. As utopian as it sounds, there will still be plenty of tasks to keep humans busy.

Obviously, the alternative is a scenario reminiscent of an Elysium-like society, where AI-owning elites jet off to space, leaving the dying planet for the rest of us, the riff-raff, to fight for dwindling resources.

Re: GPT-4

#847

Post by usaar333 »

Imnimo wrote: post #218

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…

I had worried your word "safely" biased it to using conventional safety rules that goats can't be left with cabbage, but even omitting that, it fails.

FWIW, full word substitution passes somewhat in GPT-4 (unlike GPT3), even if I heavily alter the vehicle and destinations. The problem here is that the probabilities of this being the standard puzzle are so high that your altered language isn't breaking the prior.

Re: GPT-4

#848

Post by hackernewds »

mk_stjames wrote: post #577

I just finished reading the 'paper' and I'm astonished that they aren't even publishing the # of parameters or even a vague outline of the architecture changes. It feels like such a slap in the face to all the academic AI researchers that their work is built off over the years, to just say 'yeah we're not telling you how any of this is possible because reasons'. Not even the damned parameter count. Christ.

Ironic their name is OpenAI to imply and borrow from the toils of previous academics

Re: GPT-4

#849

Post by throwaway2037 »

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

"Second Opinion machine" -- that's a good phrase. Before I read your post, the best term I heard was "summary machine". A huge part of "office work" (services) is reading and consuming large amounts of information, then trying to summarise or reason about it. Often, you are trying to find something that doesn't fit the expected pattern. If you are a lawyer, this is absolutely the future of your work. You write a short summary of the facts of the case, then ask GPT to find related case law and write the initial report. You review and ask GPT to improve some areas. It sounds very similar to how a senior partner directs their juniors, but the junior is replaced by GPT.

In my career, I saw a similar pattern with data warehouse users. Initially, managers asked junior analysts to write SQL. Later, the tools improved, and more technical managers could use a giant pivot table. Underneath, the effective query produced by the pivot table is way more complex than their previous SQL queries. Again, their jobs will change when on-site GPT become possible, so GPT can navigate their data warehouse.

It is 2023 now, and GPT-3 was already pretty good. GPT-4 will probably blow it away. What it look like in 2030? It is terrifying to me. I think the whole internet will be full of GPT-generated ad-copy that no one can distinguish from human-written material. There are a huge number of people employed as ad-copy writers on these crap ad-driven websites. What is their future work?

Re: GPT-4

#850

Post by hackernewds »

soheil wrote: post #752
mk_stjames wrote: post #577

I just finished reading the 'paper' and I'm astonished that they aren't even publishing the # of parameters or even a vague outline of the architecture changes. It feels like such a slap in the face to all the academic AI researchers that their work is built off over the years, to just say 'yeah we're not telling you how any of this is possible because reasons'. Not even the damned parameter count. Christ.

I wouldn't be surprised if this is due do some national security concerns and if the government has already been involved in every aspect of what OpenAI is doing.

Highly unlikely
Post reply on HN