Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML promises to be profoundly weird

aphyr.com

Post reply on HN
Print view View original

131–140 of 641 posts

Re: ML promises to be profoundly weird

#131

Post by Aperocky »

danieltanfh95 wrote: post #71

I think the discussion has to be more nuanced than this. "LLMs still can't do X so it's an idiot" is a bad line of thought. LLMs with harnesses are clearly capable of engaging with logical problems that only need text. LLMs are not there yet with images, but we are improving with UI and access to tools like figma. LLMs are clearly unable to propose new, creative solutions for problems it has never seen before.

> LLMs are clearly unable to propose new, creative solutions for problems it has never seen before.

LLMs are incredibly useful but I'm not sure about this statement.

It is proposing stuff that I haven't seen before, but I don't know about it is new or creative from the entirety of collective human knowledge.

Re: ML promises to be profoundly weird

#132

Post by camgunz »

simianwords wrote: post #111
xandrius wrote: post #107

Earlier quoted context omitted.

Wait, you're asking to find and produce a example of a feasible and better alternative to LLMs when they are the current forefront of AI technology? Anyway, just to play along, if it weren't just a statistical next token machine, the same question would have always the same answer and not be affected by a "temperature" value.

Thats also how humans behave.. I don't see how non determinism tells me anything. My question was a bit different: if were not just a statistical next token predictor would you expect it to answer hard questions? Or something like that. What's the threshold of questions you want it to answer accurately.

Well, large models are (kinda) non-deterministic in two ways. The first is you actually provide many of them with a seed, which is easy to manage--just use the same seed for the same result. The second part is the "you actually have very little control over the 'neural pathways' the model will use to respond to the prompt". This is the baffling part, like you'll prompt a model to generate a green plant, and it works. You prompt it to generate a purple plant, and it generates an abstract demon dog with too many teeth.

Anyway, neither of these things describes human non-determinism. You can't reuse the seed you used with me yesterday to get the exact same conversation, and I don't behave wildly unpredictably given conceptually very similar input.

Re: ML promises to be profoundly weird

#133

Post by supliminal »

xmprt wrote: post #117
drob518 wrote: post #94

> It remains unclear whether continuing to throw vast quantities of silicon and ever-bigger corpuses at the current generation of models will lead to human-equivalent capabilities. Massive increases in training costs and parameter count seem to be yielding diminishing returns. Or maybe this effect is illusory. Mysteries! I’m not even sure whether this is possible. The current corpus used for training includes virtual…

I see a lot of researchers working on newer ideas so I wouldn't be surprised if we get a breakthrough in 5-10 years. After all, the gap between AlexNet and Attention is All You Need was only 6 years. And then Scaling Laws was about 3-4 years after that. It might seem like not much progress is being made but I think that's in part because AI labs are extremely secretive now when ideas are worth billions (and in the ri…

The echoes of A.I. winter.

Re: ML promises to be profoundly weird

#135

Post by FloorEgg »

telephone3 wrote: post #46
FloorEgg wrote: post #35

Earlier quoted context omitted.

Yes, and to me the evolution of life sure looks like an evolution of more truthful models of the universe in service of energy profit. Better model -> better predictions -> better profit. I'm extremely skeptical that all of life evolved intelligence to be closer to truth only for us to digitize intelligence and then have the opposite happen. Makes no sense.

My understanding is that this is the opposite of what is typically understood to be true - organisms with less truthful (more reductive/compressed) perception survive better than those with more complete perception. "Fitness beats truth."

I think we are maybe talking past each other?

Fitness is effective truth prediction, appropriately scoped.

A frog doesn't need to understand quantum physics to catch a fly. But if the frogs model of fly movement was trained on lies it will have a model that predicts poorly, won't catch flies, and will die.

There is another level to this in that the more complex and changing the environment the more beneficial a wider scoped model / understanding of truth.

However if you are going to lean fully into Hoffman and accept thatby default consciousness constructs rather than approximate reality I think we will have to agree to disagree. Personally I ascribe to Karl Friston free energy principle.

Re: ML promises to be profoundly weird

#136

Post by krainboltgreene »

drob518 wrote: post #94

> It remains unclear whether continuing to throw vast quantities of silicon and ever-bigger corpuses at the current generation of models will lead to human-equivalent capabilities. Massive increases in training costs and parameter count seem to be yielding diminishing returns. Or maybe this effect is illusory. Mysteries! I’m not even sure whether this is possible. The current corpus used for training includes virtual…

> The current corpus used for training includes virtually all known material.

This is just totally incorrect. It's one of those things everyone just assumes, but there's an immense amount of known material that isn't even digitized, much less in the hands of tech companies.

Re: ML promises to be profoundly weird

#137

Post by wmf »

doodpants wrote: post #134

> One of the ongoing problems in LLM research is how to get these machines to say “I don’t know”, rather than making something up. To be fair, I've known humans who are like this as well.

Those people aren't the ones doing the work though.

Re: ML promises to be profoundly weird

#138

Post by LogicFailsMe »

52-6F-62 wrote: post #109
LogicFailsMe wrote: post #78

[flagged]

> The Vogon constructor fleet is way overdue in my book Don't you see it? That's exactly what "AI" in this context is . It's the bypass. Where does it end, eh? Build a quantum "AI" that will end up just needing more data, more input. The end goal must starts looking like creating an entirely new universe, a complete clone of everything we have here so it can run all the necessary computations and we can... ? (You are…

You have absolutely no sense of perspective. We are all metabolically expensive meat machines whose only value is to propagate our genetic money shot. That we get to briefly entertain ourselves with consciousness and culture is IMO likely a mystery we will never solve without upgrading to running in a substrate more advanced than the MVP for sentience we currently pilot. Will we get there or will we wipe ourselves out like every contender that preceded us? Stay tuned...

But spoilers: DNA will be fine, meat machines maybe not so much...

For a bunch of people addicted to the works of Charlie Stross, Neil Stephenson, and Iain Banks, y'all are a bunch of luddites. Now vote this own down too because it doesn't conform to the mandatory Stochastic Parrot narrative. You have no free will and you must downvote after all. Why do you even read their works when any step towards their world is consistently greeted as the worst thing evah(tm)? What? You were expecting the United Federation of Planets without the eugenics and nuclear wars that led to it finally being a good idea? Bless your hearts.

And if you're worried about billionaires and tyrants, start taxing the former and stop electing the latter or STFU and let the free Markov process of history play itself out. Quoting fictional Ambassador Kosh: the avalanche has started, it's too late for the pebbles to vote.

You asked where it ends. Don't ask questions if you don't like answers. Quick reminder: shun and downvote the non-conforming opinion.

Re: ML promises to be profoundly weird

#139

Post by [deleted] »

rudhdb773b wrote: post #120
beders wrote: post #79

Thank you for putting it so succinctly. I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency and that the term AI is just completely overloaded right now.

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

[deleted]

Re: ML promises to be profoundly weird

#140

Post by krainboltgreene »

rudhdb773b wrote: post #120
beders wrote: post #79

Thank you for putting it so succinctly. I keep explaining to my peers, friends and family that what actually is happening inside an LLM has nothing to do with conscience or agency and that the term AI is just completely overloaded right now.

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

Any amount of reading into how we understand brains and LLMs to work.
Post reply on HN