Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extracting concepts from GPT-4

openai.com

Post reply on HN
Print view View original

61–70 of 155 posts

Re: Extracting concepts from GPT-4

#61

Post by Sharlin »

TrainedMonkey wrote: post #52
thegrim33 wrote: post #15

Earlier quoted context omitted.

From the article: "We currently don't understand how to make sense of the neural activity within language models." "Unlike with most human creations, we don’t really understand the inner workings of neural networks." "The [..] networks are not well understood and cannot be easily decomposed into identifiable parts" "[..] the neural activations inside a language model activate with unpredictable patterns, seemingly re…

I read this as "we have not built up tools / math to understand neural networks as they are new and exciting" and not as "neural networks are magical and complex and not understandable because we are meddling with something we cannot control". A good example would be planes - it took a long while to develop mathematical models that could be used to model behavior. Meanwhile practical experimentation developed decent…

"We don't know how X works" literally means "we don't have models yet that can explain X's behavior".

TFA is about making a tiny bit of progress towards such models. Perhaps you should read it.

Re: Extracting concepts from GPT-4

#62

Post by Grimblewald »

itissid wrote: post #29

Does this mean that it could be a good practice to release the auto encoder that was trained on a neural network to explain its outputs? Like all open models in hugging face could have this as a useful accompaniment?

I imagine such an encoder would be specific to a model.

Re: Extracting concepts from GPT-4

#63

Post by bigyikes »

surfingdino wrote: post #51
ben_w wrote: post #41

Earlier quoted context omitted.

LLMs aren't the only kind of AI, just one of the two current shiny kinds. If a "cure for cancer" (cancer is not just one disease so, unfortunately, that's not even as coherent a request as we'd all like it to be) is what you're hoping for, look instead at the stuff like AlphaFold etc.: https://en.wikipedia.org/wiki/AlphaFold I don't know how to tell where real science ends and PR bluster begins in such models, though…

[flagged]

Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit?

This perspective discounts two important things:

1. All the things it can obviously do very well today

2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod)

I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your take. Do you not recognize that there is utility to current models? What makes it all bullshit?

Re: Extracting concepts from GPT-4

#64

Post by 3abiton »

andreyk wrote: post #12

Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows. Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from Op…

But even with current efforts so far, I don't think we have an understanding of how/why these emergent capabilities are formed. LLMs are still a black box as ever.

Re: Extracting concepts from GPT-4

#67

Post by choppaface »

andreyk wrote: post #12

Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows. Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from Op…

The Deep Visualization Toolbox from nearly 10 years ago is solid precedent for understanding deep models, albeit much smaller models than LLMs. It’s hard to say OpenAI’s “visualization” released today is nearly as effective. It could be that GPT-4 is much harder to instrument.

https://github.com/yosinski/deep-visualization-toolbox

Re: Extracting concepts from GPT-4

#69

Post by atleastoptimal »

jackallis wrote: post #68

"We currently don't understand how to make sense of the neural activity within language models" this is why peopl are up-in-arms.

Up in arms for what reason? That neural networks are perfectly interpretable? That's the nature of huge amorphous deep networks. These feature extraction forays are a good step forward though.

Re: Extracting concepts from GPT-4

#70

Post by baobabKoodaa »

calibas wrote: post #43
yorwba wrote: post #27

Earlier quoted context omitted.

That's why they have buttons to choose which model's tokenizer to use.

Yes, thank you, I understand that part. It's the might condition in the description that makes me think the results might not be the exact same as what's used in the live models.

The results are the same.
Post reply on HN