Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extracting concepts from GPT-4

openai.com

Post reply on HN
Print view View original

111–120 of 155 posts

Re: Extracting concepts from GPT-4

#111

Post by irthomasthomas »

localfirst wrote: post #30
OtherShrezzing wrote: post #28

Earlier quoted context omitted.

LLM based AIs have lots of "features" which are kind of synonymous with "concepts" - these can be anything from `the concept of an apostrophe in the word don't`, to `"George Wash" is usually followed by "ington" in the context of early American History`. Inside of the LLMs neural network, these are mapped to some circuitry-in-software-esque paths. We don't really have a good way of understanding how these features ar…

hallucination solution?

Aye, but where only openai gets to decide what is hallucination.

Re: Extracting concepts from GPT-4

#112

Post by realPtolemy »

lanceflt wrote: post #103
realPtolemy wrote: post #99

Earlier quoted context omitted.

Indeed, and the very last section about how they’ve now “open sourced” this research is also a bit vague. They’ve shared their research methodology and findings… But isn’t that obligatory when writing a public paper?

https://github.com/openai/sparse_autoencoder They actually open sourced it, for GPT-2 which is an open model.

Thanks, I must have read through the document to hastily.

Re: Extracting concepts from GPT-4

#113

Post by ben_w »

surfingdino wrote: post #110
ben_w wrote: post #100

Earlier quoted context omitted.

Your first section is very much a limit of LLMs, but again, that's not all AI — if you want an AI to play chess, and you want to actually win, you use Stockfish or AlphaZero, because if you use an LLM it will perform illegal moves.

Why would I want to use an AI to win a game of chess? Where's the fun and challenge in it? "Go win me a chess tournament" is the wish nobody has unless we are talking about someone who wants to pretend to be a chess master. It's still a small market. Examples like these are very common in the AI community, they are solutions to problems nobody has.

> Why would I want to use an AI to win a game of chess?

https://www.bbc.com/news/world-63043023

Though such a question misses the point: use the right tool for the job.

(For a non-ML example, I don't know why 5/8 wrenches exist, but I'm confident of two things: (1) that's a very specific size, unlikely to be useful for a different sized problem; (2) using one as a hammer would be sub-optimal).

I'm not interested in creating a taxonomy of special purpose AI models which each do one thing well and nothing else. What I can do is give a handful of famous examples, such as chess.

Other choices at my disposal (but purely off the top of my head and in no way systematic) include the use of OCR to read and hence sort post faster than any human and also more accurately than all but the very best. Or in food processing for quality control (I passed up on a student work placement for that 20 years ago). Or the entirity of Google search, Gmail's spam filters, Maps' route finding and at least some of its knowledge of house numbers, their CAPTCHA system, and Translate (the one system on this list which is fundamentally the same an LLM). Or ANPR.

It's like you're saying "food is bad" because you don't like cabbage — the dislike is a valid preference, of course it is, but it doesn't lead to the conclusion.

Re: Extracting concepts from GPT-4

#114

Post by ben_w »

realPtolemy wrote: post #101
thegrim33 wrote: post #15

Earlier quoted context omitted.

From the article: "We currently don't understand how to make sense of the neural activity within language models." "Unlike with most human creations, we don’t really understand the inner workings of neural networks." "The [..] networks are not well understood and cannot be easily decomposed into identifiable parts" "[..] the neural activations inside a language model activate with unpredictable patterns, seemingly re…

Could there also be a “legal hedging” reason for why you would release a paper like this? By reaffirming that “we don’t know how this works, nobody does” it’s easier to avoid being charged with copyright infringement from various actors/data sources that have sued them.

I'd be surprised if doing so had any impact on the lawsuits, but I'm not a lawyer.

Re: Extracting concepts from GPT-4

#115

Post by ben_w »

ein0p wrote: post #91
submeta wrote: post #90

Earlier quoted context omitted.

Scary actually. Because how can we asses the risks when we don’t know what the system is capabale of doing.

We know exactly what the system is capable of doing. It’s capable of outputting tokens which can then be converted into text.

Which is so broad as to be unhelpful.

We also know that petroleum mixed with air may be combusted to release energy; we needed to characterise this much better in order for the motor car to be distinguishable from a fuel-air bomb.

Re: Extracting concepts from GPT-4

#116

Post by joaquincabezas »

leogao wrote: post #77
andreyk wrote: post #12

Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows. Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from Op…

We were planning to release the paper around this time independent of the other events you mention. I think it is still predominantly accurate to say that we have no idea how LLMs work. SAEs might eventually change that, but there's still a long way to go.

it makes sense that the leaders are building around similar ideas in parallel, for me it's a healthy sign

Re: Extracting concepts from GPT-4

#117

Post by baxtr »

gradus_ad wrote: post #59
TrainedMonkey wrote: post #52

Earlier quoted context omitted.

I read this as "we have not built up tools / math to understand neural networks as they are new and exciting" and not as "neural networks are magical and complex and not understandable because we are meddling with something we cannot control". A good example would be planes - it took a long while to develop mathematical models that could be used to model behavior. Meanwhile practical experimentation developed decent…

Chaotic nonlinear dynamics have been an object of mathematical research for a very long time and we have built up good mathematical tools to work with them, but in spite of that turbulent flow and similar phenomena (brains/LLM's) remain poorly understood. The problem is that the macro and micro dynamics of complex systems are intimately linked, making for non-stationary non-ergodic behavior that cannot be reduced to…

Physicists would probably argue that the system might be understood but that we don’t have the model for it yet.

Many natural phenomena look chaotic at best without a model. Once you have a model things fall into place and everything starts looking orderly.

Maybe it cannot be reduced. But maybe we are just observing the peripherals without understanding the inner workings.

Re: Extracting concepts from GPT-4

#118

Post by passwordoops »

TrainedMonkey wrote: post #52
thegrim33 wrote: post #15

Earlier quoted context omitted.

From the article: "We currently don't understand how to make sense of the neural activity within language models." "Unlike with most human creations, we don’t really understand the inner workings of neural networks." "The [..] networks are not well understood and cannot be easily decomposed into identifiable parts" "[..] the neural activations inside a language model activate with unpredictable patterns, seemingly re…

I read this as "we have not built up tools / math to understand neural networks as they are new and exciting" and not as "neural networks are magical and complex and not understandable because we are meddling with something we cannot control". A good example would be planes - it took a long while to develop mathematical models that could be used to model behavior. Meanwhile practical experimentation developed decent…

"neural networks as they are new"

Yup, ANNs have only been around since the 1950s... Brand spanking new

Re: Extracting concepts from GPT-4

#119

Post by ben_w »

vitus wrote: post #73
bigyikes wrote: post #63

Earlier quoted context omitted.

Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your…

> 1. All the things it can obviously do very well today I'm curious what those things are. At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more). I don't have a good sense of when they fail, how often this happens, or how to…

> But when you start talking about emergent behavior from machine learning, the failure modes are much harder to reason about.

Sure, this is basically why so many are concerned AI might kill us all:

Lots of observed emergent phenomena that are not expected (basically everything ChatGPT can do given it was trained on next token prediction), as a result of doing exactly what we said instead of what we meant (all computer bugs ever), doing it so hard that something breaks (Goodhart's law), doing it so fast that humans can't respond (stock market flash-crashes, many robotic control systems), and being so capable in smaller scale tests that they tempt people to let go of the metaphorical steering wheel (the lawyers citing ChatGPT, but also previously a T-shirt company that dictionary merged verbs into "keep calm and …" without checking, and even more previously either Amazon or eBay dictionary merging nouns into "${x}: buy it cheap on {whichever site it was}" with nouns including "plutonium" and "slaves").

If people had a good model for the AI, it wouldn't be a problem, we'd simply use them only for what they are good at and nothing else.

Re: Extracting concepts from GPT-4

#120

Post by surfingdino »

sroussey wrote: post #92
vitus wrote: post #73

Earlier quoted context omitted.

> 1. All the things it can obviously do very well today I'm curious what those things are. At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more). I don't have a good sense of when they fail, how often this happens, or how to…

> I'm curious what those things are. > At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more) That’s a list of things that gets clicks in the popular press. Some solutions I love: Recording a video and creating a transcript f…

You may want to familiarise yourself with https://www.vice.com/en/article/gvy4gb/one-mans-david-and-go...
Post reply on HN