[flagged]
ML promises to be profoundly weird
551–560 of 641 posts
Re: ML promises to be profoundly weird
#552Earlier quoted context omitted.
It's sort of the exception that proves the rule. This is where STEM people are weak- a lack of knowledge on history. In another forum, someone would have chipped in that England's virgin forests were fully deforested by 1150. And someone else would have pointed out that this deforestation produced the economic demand for coal that drove the Industrial Revolution in the first place. Still, that kind of underscores OP'…
> powered solely by human muscle Both animals and water power go way back. The early steam engine was measured in horsepower because that’s what it was replacing in mines. It couldn’t compete with nearby water power which was already being moved relatively long distances through mechanical means at the time. Hand waving this as unimportant really misunderstands just how limited the Industrial Revolution was.
“The industrial revolution wasn’t really all that” is such a strange hill to die on.
Re: ML promises to be profoundly weird
#553Earlier quoted context omitted.
Structurally a transformer model is so unrelated to the shape of the brain there's no reason to think they'd have many similarities. It's also pretty well established that the brain doesn't do anything resembling wholesale SGD (which to spell it is evidence that it doesn't learn in the same way).
If platonic representation hypothesis holds across substrates, then it might matter very little, in the end. It holds across architectures in ML, empirically. The crowd of "backpropagation and Hebbian learning + predictive coding are two facets of the very same gradient descent" also has a surprisingly good track record so far.
I don't know which direction you're going with this, but predictive coding has a pretty obvious advantage when it comes to continuous learning. Since predictive coding primarily encodes errors, it can distinguish between known and novel data and therefore reduce the damaging effects of catastrophic forgetting by having a very obvious regularisation scheme for avoiding forgetting.
Re: ML promises to be profoundly weird
#554[flagged]
Re: ML promises to be profoundly weird
#555I think the discussion has to be more nuanced than this. "LLMs still can't do X so it's an idiot" is a bad line of thought. LLMs with harnesses are clearly capable of engaging with logical problems that only need text. LLMs are not there yet with images, but we are improving with UI and access to tools like figma. LLMs are clearly unable to propose new, creative solutions for problems it has never seen before.
> LLMs with harnesses are clearly capable of engaging with logical problems that only need text. > LLMs are clearly unable to propose new, creative solutions for problems it has never seen before. How do you reconcile this with this article that the author linked? It's not a novel problem, and it's only text: https://medium.com/the-generator/one-word-answers-expose-ai-... I guess it's a form of engagement to give a w…
The author of the medium article specifically hobbled the models to stop them thinking it through and got a wrong answer but that would happen with humans too and doesn't prove much.
Re: ML promises to be profoundly weird
#556[dead]
Re: ML promises to be profoundly weird
#557Nice, can't view it.
"Unavailable Due to the UK Online Safety Act"
Re: ML promises to be profoundly weird
#558I think it's too early to declare the Turing test passed. You just need to have a conversation long enough to exhaust the context window. Less than that, since response quality degrades long before you hit hard window limits. Even with compaction. Neuroplasticity is hard to simulate in a few hundred thousand tokens.
>Consider first the more accurate form of the question. I believe that
in about fifty years' time it will be possible, to programme computers, with
a storage capacity of about 10^9, to make them play the imitation game so
well that an average interrogator will not have more than 70 per cent chance
of making the right identification after five minutes of questioning. (Turing 1950)
That was the test as discussed by Turning - five minutes, It's not that demanding. The test you mention could maybe be called an enhanced Turing test but the original one is pretty much passed.
He was a bit off on the time taken and memory used. I think more like 75 years and 50 GB rather than 50 years and 125 MB.
Re: ML promises to be profoundly weird
#559Earlier quoted context omitted.
> They do not reason How do you disprove it?
We know that they do not reason because we know the algorithm behind the curtain. The model is generating the next token via model weights and some randomness. That’s all. It not reasoning. Sometimes it has an appearance of reasoning, but not if you know how it works. It doesn’t matter that the model manufacturer marketing department slaps a “Reasoning!” sticker on the side of the model. It’s not actually doing that.…
what if eventually the physical mechanism for human consciousness becomes fully understood? does understanding that process mean we are no longer intelligent?
Re: ML promises to be profoundly weird
#560Earlier quoted context omitted.
I see a lot of researchers working on newer ideas so I wouldn't be surprised if we get a breakthrough in 5-10 years. After all, the gap between AlexNet and Attention is All You Need was only 6 years. And then Scaling Laws was about 3-4 years after that. It might seem like not much progress is being made but I think that's in part because AI labs are extremely secretive now when ideas are worth billions (and in the ri…
The echoes of A.I. winter.
Yep, last time we got "a lot of researchers working on newer ideas", it took them 20 years to get into a working idea, and other 20 years to get it mature enough to make an AI boom.