Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

socraticmodels.github.io

Post reply on HN
Print view View original

11–20 of 40 posts

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#11

Post by nynx »

vagabund wrote: post #10
nynx wrote: post #9

Earlier quoted context omitted.

I’m certainly not discounting the results and I don’t see anything wrong with suggesting what I think would generally be a good path to look at in the future.

It's not wrong per se, and I'm obviously in no place to police the discussion, but it's only tangentially related to the post and often clouds out what would be a more pointed deliberation over this research. Maybe I'm expecting too much of HN, but I've seen these same two top level comments under myriad ML posts. Sorry for the meta-discussion that's gotten us further away from this really remarkable paper.

Point taken, I do agree with you that it’s probably best to stay on topic in these kinds of posts.

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#12

Post by ceeplusplus »

vagabund wrote: post #8
nynx wrote: post #4

This is super impressive. Transformers have consistently done better than almost anyone thought. I still hold the opinion that we’re going to need to move to spiking neuron (SNN) models in the future to keep growing the networks. Spiking networks require lots of storage, but a lot, lot less compute. They also propagate additional information in the _timing_ of the spikes, not just the values. There are a lot of low-h…

The comments of every ML paper posted on this site are dominated by people either baselessly discounting the results as a party trick or illusion, or shoehorning in their conjecture about what approach the field is overlooking. As someone just trying to learn more about the implications of new research, I find myself resorting to /r/machinelearning, or even twitter threads, to get timely and informed discussions. Tha…

As a community grows it attracts people who don't have the same background that drew the original members of the community together, so it becomes inevitable to see this kind of layman commentary. I've seen it happen to r/hardware which has been taken over by gamers with no CS background and AMD shareholders when it used to have a lot of knowledgable people commenting.

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#13

Post by mountainriver »

vagabund wrote: post #8
nynx wrote: post #4

This is super impressive. Transformers have consistently done better than almost anyone thought. I still hold the opinion that we’re going to need to move to spiking neuron (SNN) models in the future to keep growing the networks. Spiking networks require lots of storage, but a lot, lot less compute. They also propagate additional information in the _timing_ of the spikes, not just the values. There are a lot of low-h…

The comments of every ML paper posted on this site are dominated by people either baselessly discounting the results as a party trick or illusion, or shoehorning in their conjecture about what approach the field is overlooking. As someone just trying to learn more about the implications of new research, I find myself resorting to /r/machinelearning, or even twitter threads, to get timely and informed discussions. Tha…

As an ML engineer I found the comment insightful. I agree HN takes a critical approach to list ML but that’s largely because there’s been so much snake oil with it

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#14

Post by mountainriver »

arjvik wrote: post #2

We've come to the consensus that large language models are just stochastic parrots... What makes us think that we can achieve a higher level of intelligence by putting them in conversation? I think the next step in NLP will be a drastic innovation on today's learning model.

We have not come to that consensus and large language models display really interesting capabilities like few shot learning, which before we thought would require a widely different architecture

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#16

Post by nynx »

ceeplusplus wrote: post #12
vagabund wrote: post #8

Earlier quoted context omitted.

The comments of every ML paper posted on this site are dominated by people either baselessly discounting the results as a party trick or illusion, or shoehorning in their conjecture about what approach the field is overlooking. As someone just trying to learn more about the implications of new research, I find myself resorting to /r/machinelearning, or even twitter threads, to get timely and informed discussions. Tha…

As a community grows it attracts people who don't have the same background that drew the original members of the community together, so it becomes inevitable to see this kind of layman commentary. I've seen it happen to r/hardware which has been taken over by gamers with no CS background and AMD shareholders when it used to have a lot of knowledgable people commenting.

I don't claim to be an expert, but I actually do undergraduate neuromorphic computing research. So, I don't know much, but I do know a little about what I'm talking about.

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#17

Post by gjm11 »

arjvik wrote: post #2

We've come to the consensus that large language models are just stochastic parrots... What makes us think that we can achieve a higher level of intelligence by putting them in conversation? I think the next step in NLP will be a drastic innovation on today's learning model.

"Stochastic parrots" -- have you seen, e.g., the examples in the PaLM paper of how it does on "chained inference" tasks? I don't see how you can classify that as mere parroting.

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#18

Post by robbedpeter »

mountainriver wrote: post #15

This is really awesome, multimodal is definitely where transformers are headed and holds the promise of solving a lot of the grounding issues we see with the current sota

Elon's robots might actually work out, at least in software.

This type of methodology, doing meta-cognitive programming by linking together different models, is awesome. They're constructing low resolution imitations of brains - gpt-3 and BERT and the like can do things that no individual model can achieve. A predicate logic layer can document and explain decision history, and the other modules start to resemble something like the subconscious mind.

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#19

Post by gwern »

vagabund wrote: post #8
nynx wrote: post #4

This is super impressive. Transformers have consistently done better than almost anyone thought. I still hold the opinion that we’re going to need to move to spiking neuron (SNN) models in the future to keep growing the networks. Spiking networks require lots of storage, but a lot, lot less compute. They also propagate additional information in the _timing_ of the spikes, not just the values. There are a lot of low-h…

The comments of every ML paper posted on this site are dominated by people either baselessly discounting the results as a party trick or illusion, or shoehorning in their conjecture about what approach the field is overlooking. As someone just trying to learn more about the implications of new research, I find myself resorting to /r/machinelearning, or even twitter threads, to get timely and informed discussions. Tha…

Don't forget /r/mlscaling!

Re: Socratic Models – Composing Zero-Shot Multimodal Reasoning with Language

#20

Post by robbedpeter »

arjvik wrote: post #2

We've come to the consensus that large language models are just stochastic parrots... What makes us think that we can achieve a higher level of intelligence by putting them in conversation? I think the next step in NLP will be a drastic innovation on today's learning model.

There is no such consensus. Transformers navigate problem spaces with various mechanisms that include recursion, and multi-pass inference means the depth can be arbitrary. This means that models pick up on the functions that generate answers, not simple statistical relationships you see in Markov chains.

"Stochastic parrot" is a derogatory term and I've never seen anyone who actually understands the technology use that phrase unironically. If anything, it's a shibboleth for bias or ignorance.

Post reply on HN