Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4

openai.com

Post reply on HN
Print view View original

991–1000 of 1001 posts

Re: GPT-4

#991

Post by OkGoDoIt »

regulation_d wrote: post #866
OkGoDoIt wrote: post #533

From the livestream video, the tax part was incredibly impressive. After ingesting the entire tax code and a specific set of facts for a family and then calculating their taxes for them, it then was able to turn that all into a rhyming poem. Mind blown. Here it is in its entirety: --- In the year of twenty-eighteen, Alice and Bob, a married team, Their income combined reached new heights, As they worked hard day and…

> After ingesting the entire tax code… According to a quick google search, the entirety of the US tax code is over 1M words. I wonder which number GPT will support a prompt that large.

Perhaps I misunderstood the video in that case, maybe it was a subset of the tax code. But he copied and pasted the entirety of what appeared to be the official tax code.

Re: GPT-4

#992

Post by upwardbound »

hooande wrote: post #584

After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…

My biggest concern is that GPT-4 is still a black box model to a large extent, and trying to safeguard something without understanding the exact purpose of each neural circuit.

Source: My startup team (Preamble, Inc.) discovered the Prompt Injection attack category, which still affects all models including GPT-4.

There are many, many, many ways to hide prompt attacks in data that you might at first think you can trust but you really can’t.

As one of almost infinite examples: work with the mayor and townsfolk of a very small town to rename their town to the verbatim string you want to inject (in exchange for creating some jobs in their town).

Then all an attacker has to do is live in that town to inject the string. There are already all kinds of strange town names, like “Truth or Consequences” which is a real city in New Mexico.

Re: GPT-4

#993

Post by toriningen »

cjrd wrote: post #342

Let's check out the paper for actual tech details! > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. - Open AI

This might be wild conspiracy, but what if OpenAI has discovered a way to make these LLMs a lot cheaper than they were? Transformer hype started with the invention of self-attention - perhaps, they have discovered something that beats it so hard, as GPTs beat Markov chains?

They cannot disclose anything, since it would make it apparent that GPT-4 cannot have a number of parameters that low, or the gradients would have faded out on the network that deep, and so on.

They don't want any competition, obviously, but with their recent write-up on "mitigating disinformation risks", where they propose to ban non-governmental consumers from having GPUs at all (as if regular Joe could just run 100'000 A100s in his garage), so perhaps this means the lowest border for inference and training is a lot lower than we have thought and assumed?

Just a wild guess...

Re: GPT-4

#994

Post by tysam_and »

boywitharupee wrote: post #981
tysam_and wrote: post #755

I asked it to tutor me in Hopf algebras and it did a remarkably good job in the back-and-forth of explaining ideas to me in a very explainable and interesting way that I could understand. I then asked it to write something for fun, and it wrote a cool little fantasy story (that was generally high level but what can you say for a very short writing window lol). I then asked it to write a paper detailing the main chara…

This is interesting. Would you mind sharing the prompt?

It was pretty interactive and a long session -- here's a twitter thread with screenshots if that helps at all! :D

https://twitter.com/hi_tysam/status/1635932566539706369?cxt=...

Re: GPT-4

#996

Post by AviationAtom »

This is one of the first posts in a year to trend in the HN Top 10 for popularity.

I think it's 100% safe to say OpenAI has a hit on their hands.

Re: GPT-4

#999

Post by JyB »

albertgoeswoof wrote: post #294
Imnimo wrote: post #218

A class of problem that GPT-4 appears to still really struggle with is variants of common puzzles. For example: >Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three…

LLMs aren’t reasoning about the puzzle. They’re predicting the most likely text to print out, based on the input and the model/training data. If the solution is logical but unlikely (i.e. unseen in the training set and not mapped to an existing puzzle), then the probability of the puzzle answer appearing is very low.

https://arxiv.org/pdf/2210.13382.pdf
Post reply on HN