Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Extracting concepts from GPT-4

openai.com

Post reply on HN
Print view View original

91–100 of 155 posts

Re: Extracting concepts from GPT-4

#91

Post by ein0p »

submeta wrote: post #90
thegrim33 wrote: post #15

Earlier quoted context omitted.

From the article: "We currently don't understand how to make sense of the neural activity within language models." "Unlike with most human creations, we don’t really understand the inner workings of neural networks." "The [..] networks are not well understood and cannot be easily decomposed into identifiable parts" "[..] the neural activations inside a language model activate with unpredictable patterns, seemingly re…

Scary actually. Because how can we asses the risks when we don’t know what the system is capabale of doing.

We know exactly what the system is capable of doing. It’s capable of outputting tokens which can then be converted into text.

Re: Extracting concepts from GPT-4

#92

Post by sroussey »

vitus wrote: post #73
bigyikes wrote: post #63

Earlier quoted context omitted.

Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your…

> 1. All the things it can obviously do very well today I'm curious what those things are. At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more). I don't have a good sense of when they fail, how often this happens, or how to…

> I'm curious what those things are.

> At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more)

That’s a list of things that gets clicks in the popular press.

Some solutions I love:

Recording a video and creating a transcript from it. Then edit the transcript and the video gets edited.

Translating (dubbing) video and changing the lips of the speaker to match where they should be for the new audio.

Scanning invoices for mistakes (I know entire businesses build just in this one thing).

Understanding edge cases. So many things have some nice hard and fast rules that can fail, and an ML system of some sort can figure it out so the system can move on. I do this for processing SEC data, and soon for web scraping (like they change the html but visually it’s kinda the same, and the ai system can figure it, give me a new html selector, and back in business).

Re: Extracting concepts from GPT-4

#93

Post by reducesuffering »

ein0p wrote: post #91
submeta wrote: post #90

Earlier quoted context omitted.

Scary actually. Because how can we asses the risks when we don’t know what the system is capabale of doing.

We know exactly what the system is capable of doing. It’s capable of outputting tokens which can then be converted into text.

And social media manipulation is just registers and bytes, wait no, sand and electrons.

Re: Extracting concepts from GPT-4

#94

Post by darby_nine »

andreyk wrote: post #12

Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows. Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from Op…

> Mapping the Mind of a Large Language Model

The fact that a paper is implying a LLM has a mind doesn't exactly bode well for the people who wrote it, not to mention the continued meaningless babbling about "safety". It'd also be nice if they could show their work so we could replicate it. Still, not shabby for an ad!

Re: Extracting concepts from GPT-4

#95

Post by surfingdino »

bigyikes wrote: post #63
surfingdino wrote: post #51

Earlier quoted context omitted.

[flagged]

Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your…

> Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit?

It is sold as a research tool, but it cannot be trusted to return facts, because it will happily recombine disconnected pieces of data. AI cannot tell truth from lies, it is good at constructing output that looks like an answer but it does not care about the factual correctness. Google search result summaries are a good example of this problem. When I searched for "what happened to the inventor of Tetris?" it took bios of two Russian-born developers, one Pajitnov and another a murderer and combined them into one presenting Pajitnov as a murderer. I thought it did not sound right and did some additional searching and sure enough it wasn't true, but how many people who were shown that answer have become convinced that he was a murderer? What if his neighbours saw it? What if such made up summaries are fed into a system that decides who can board a plane? When I bring this problem up people tell me it's not an issue and you should always go back and verify facts, but what if the sources I use have the same problem of being made up content? We are not telling people to stop, think, and verify outputs produced by AI, we are telling them AI is making them "more productive" so they use it to produce garbage content without checking the facts. Please explain to me the usefulness of a tool I cannot trust? Producing garbage faster is not something I wake up in the morning wanting to do more of.

> 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod)

Unlike AI VCs can count and would like to see a return in their investments. I don't think there's much to show for it so far.

Re: Extracting concepts from GPT-4

#97

Post by dazhbog »

Is this like an fMRI for a neural net? We can see which regions light up depending on various topics..

I wonder if an assessment neural net can be plugged in to evaluate the regions that light up automatically.., just like when they had an AI reconstruct what the patient was looking at, from only fMRI scans!

Re: Extracting concepts from GPT-4

#98

Post by ben_w »

allturtles wrote: post #82
interstice wrote: post #74

Earlier quoted context omitted.

There are two mindsets at play here, the cynics vs the optimists. I’m an optimist to a fault by nature, but I also think there is a kind of Pascals bet to be played here. If you bet sensibly on the current bubble/wave and turn out to be wrong - well you’re in the same place as everyone else with maybe some time and money lost. But if you’re right things get interesting.

> But if you’re right things get interesting. In what way? If the current bubble/wave turns out to be right doesn't that mean we're all out of a job? Unless by betting on it you mean buying Nvidia stock?

We might all be out of a job if the AI reached superhuman performance at everything (or even just human at much lower cost), but even without that this can still be a 10x speedup of the rate of change in the industrial revolution.

What "out of a job" means is currently unclear: do we all get UBI, or starve? Just because we could get UBI doesn't mean we will, and the transition has to be very fast (because the software won't be good enough until one day when a software update comes) and very well managed and almost simultaneous worldwide, which probably means it will go wrong.

Re: Extracting concepts from GPT-4

#99

Post by realPtolemy »

andreyk wrote: post #12

Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows. Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from Op…

Indeed, and the very last section about how they’ve now “open sourced” this research is also a bit vague. They’ve shared their research methodology and findings… But isn’t that obligatory when writing a public paper?

Re: Extracting concepts from GPT-4

#100

Post by ben_w »

surfingdino wrote: post #95
bigyikes wrote: post #63

Earlier quoted context omitted.

Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your…

> Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? It is sold as a research tool, but it cannot be trusted to return facts, because it will happily recombine disconnected pieces of data. AI cannot tell truth from lies, it is good at constructing output that looks like an answer but it does not care about the factual correctness. Google search res…

Your first section is very much a limit of LLMs, but again, that's not all AI — if you want an AI to play chess, and you want to actually win, you use Stockfish or AlphaZero, because if you use an LLM it will perform illegal moves.
Post reply on HN