Curious what interesting ways have you used chatGPT ... yesterday i used it to test my pool levels (took a pic of my pool strip i had laying around and told it the brand of pool strip).
Extracting concepts from GPT-4
81–90 of 155 posts
Re: Extracting concepts from GPT-4
#82Earlier quoted context omitted.
Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your…
There are two mindsets at play here, the cynics vs the optimists. I’m an optimist to a fault by nature, but I also think there is a kind of Pascals bet to be played here. If you bet sensibly on the current bubble/wave and turn out to be wrong - well you’re in the same place as everyone else with maybe some time and money lost. But if you’re right things get interesting.
> But if you’re right things get interesting.
In what way? If the current bubble/wave turns out to be right doesn't that mean we're all out of a job? Unless by betting on it you mean buying Nvidia stock?
Re: Extracting concepts from GPT-4
#83Is anyone else weirded out that they do not acknowledge Anthropic at all?
Re: Extracting concepts from GPT-4
#84Is anyone else weirded out that they do not acknowledge Anthropic at all?
The paper cites Anthropic's work extensively.
Re: Extracting concepts from GPT-4
#85What applications do you think this opens up?
One of the first ones I can think of is pairing it up with Browser extensions to increase productivity of knowledge workers. Think of sales people quickly assessing the viability of leads, or simply reducing noise on social media as a B2C app.
Re: Extracting concepts from GPT-4
#86Earlier quoted context omitted.
Solving this problem would be a step on the way to debugging (and then resolving, or at least highlighting) hallucinations.
I’m skeptical that it could ever be possible to tell the difference between a hallucination and a “fact” in terms of what’s going on inside the model. Because hallucinations aren’t really a bug in the usual sense. Ie, there’s not some logic wrong or something misfiring. Instead, it’s more appropriate to think of LLMs as always hallucinating. And sometimes that comes really close to reality because there’s a lot of re…
> Instead, it’s more appropriate to think of LLMs as always hallucinating.
That matches my mental model as well. To get rid of hallucinations, "I don't know" would have to be an acceptable answer, and it would have to output that when 'appropriate' ... Which, it doesn't know (and to be fair, neither do we most of the time, without some way of checking/validating(.
Re: Extracting concepts from GPT-4
#87Earlier quoted context omitted.
> Wonder if this was a bit rushed out in response to Anthropic's release too lazy to dig up source but some twitter sleuth found that the first commit to the project was 6 months ago likely all these guys went to the same metaphorical SF bars, it was in the water
This project has been in the works for about a year. The initial commit to the public repo was not really closely related to this project, it was part of the release of the Transformer debugger, and the repo was just reused for this release.
ha thank you Leo; i myself felt uneasy pointing out commit date based evidence and you just proved why.
mild followup question: any alpha to be gained from training the same SAEs on two different generations of GPT4, eg GPT4 on march 2023 vs june 2023 vintage, whatever is most architecturally comparable, and diffing them. what would be your priors on what you’d find?
Re: Extracting concepts from GPT-4
#88Earlier quoted context omitted.
> Wonder if this was a bit rushed out in response to Anthropic's release too lazy to dig up source but some twitter sleuth found that the first commit to the project was 6 months ago likely all these guys went to the same metaphorical SF bars, it was in the water
Visualizer was added 18 hours ago: https://github.com/openai/sparse_autoencoder/commit/764586ae...
It’s hard to believe it was written overnight.. this seems more like a public stable dump of what they’ve been working on without saying when they started. Some clues could come from looking at when all the deps it uses were released. They’re also calling this version 0.1.67, though I’m not sure that means anything either.
Re: Extracting concepts from GPT-4
#89Earlier quoted context omitted.
Is your argument that because AI can’t currently do the arbitrary things you wish it would do, it is therefore bullshit? This perspective discounts two important things: 1. All the things it can obviously do very well today 2. Future advancements to the tech (billions are pouring in, but this takes time to manifest in prod) I’m trying not to be one of the “guys” you’re talking about, but I just can’t comprehend your…
> 1. All the things it can obviously do very well today I'm curious what those things are. At least to me, it isn't obvious that LLMs solve any of their many applications from the past year "very well". I worry about failures (hallucinations, misinterpretation of prompts, regurgitation of incorrect facts, violation of copyright, and more). I don't have a good sense of when they fail, how often this happens, or how to…
> I'm curious what those things are
I find them pretty good at reasoning about tree structures that have a depth which I myself find difficult to navigate. For instance, I've been working with libcst (a syntax tree library) and I can say:
1. observe this refactor rule which I like
2. here's the starting code
3. here's the desired code
4. write me a refactor rule, in the style of 1, which transforms 2 into 3
It sometimes takes a few iterations where I show it a diff which highlights how its rule fails to construct 3 given 2, but it usually gets me the transformation I need much faster than I'd have done so by hand.
And once I'm done I have a rule which I can apply without the LLM in the loop and which is much more robust than something like a patch file (which often fail to apply for irrelevant reasons like whitespace or comments that have changed since I wrote the rule).
The key is to find cases, like this one, where you can sort of encircle the problem with context from multiple sides, one of which works as a pass/fail indicator. Hallucinations happen, but you use that indicator to ensure that they get retried with the failure text as correcting context.
It helps to design your code so that those context pieces stay small and reasonably self-describing without taking a foray deep into the dependencies, but then that's just a good idea anyway.
Re: Extracting concepts from GPT-4
#90Exciting to see this so soon after Anthropic's "Mapping the Mind of a Large Language Model" (under 3 weeks). I find these efforts really exciting; it is still common to hear people say "we have no idea how LLMs / Deep Learning works", but that is really a gross generalization as stuff like this shows. Wonder if this was a bit rushed out in response to Anthropic's release (as well as the departure of Jan Leike from Op…
From the article: "We currently don't understand how to make sense of the neural activity within language models." "Unlike with most human creations, we don’t really understand the inner workings of neural networks." "The [..] networks are not well understood and cannot be easily decomposed into identifiable parts" "[..] the neural activations inside a language model activate with unpredictable patterns, seemingly re…
Scary actually. Because how can we asses the risks when we don’t know what the system is capabale of doing.