Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are no new ideas in AI, only new datasets

blog.jxmo.io

Post reply on HN
Print view View original

271–280 of 307 posts

Re: There are no new ideas in AI, only new datasets

#271

Post by ekunazanu »

timewizard wrote: post #232
ekunazanu wrote: post #182

Earlier quoted context omitted.

Winning two Nobel prizes wasn't enough progress?

Is progress measured in nobel prizes? My understanding is those are put to a vote by institutional committee. Putting that aside. The shared prize in 2024 was given for work done in the 1970s and 1980s. Was this meant to be a confirmation of my point? You've done so beautifully. In 2022 they saw fit to award Ben Bernanke. Yep. That one. For, I kid you not, work on the impacts of financial crises. Ironically also work…

AlphaFold uses transformers. That is definitely not from the 70s and 80s.

Progress for me includes both small iterative refinements and big leaps. It also includes trying old techniques in new domains with new technology. So I think we just have differing definitions for progress.

Re: There are no new ideas in AI, only new datasets

#272

Post by yencabulator »

dinfinity wrote: post #235
computably wrote: post #228

Earlier quoted context omitted.

Language is literally an abstraction of sensory inputs and cognitive processes. One can make similar arguments about image generation. These abstractions might characterize the higher cognitive abilities of humans, but it makes no sense to ignore "lower level" cognition. Embodiment is the foundation of our rich internal world models, in particular spacetime, causality, etc. Current generative models merely mimic the…

> Language is literally an abstraction of sensory inputs and cognitive processes. Language can exist entirely independently from senses and cognition. It is an encoding of patterns in the world where the only thing that matters is if anybody or anything wielding it can map the encodings to and from the patterns they encode for (which is more of a sociological/synchronisation challenge). Does C, or Java, 'make no sens…

> Language can exist entirely independently from senses and cognition.

Helen Keller begs to disagree. Language and cognition were clearly linked for her.

> It wasn't until April 5, 1887, when Anne took Helen to an old pump house, that Helen finally understood that everything has a name. Sullivan put Helen’s hand under the stream and began spelling “w-a-t-e-r” into her palm, first slowly, then more quickly.

> Keller later wrote in her autobiography, “As the cool stream gushed over one hand she spelled into the other the word water, first slowly, then rapidly. I stood still, my whole attention fixed upon the motions of her fingers. Suddenly I felt a misty consciousness as of something forgotten–-a thrill of returning thought; and somehow the mystery of language was revealed to me. I knew then that ‘w-a-t-e-r’ meant the wonderful cool something that was flowing over my hand. That living word awakened my soul, gave it light, hope, joy, set it free! There were barriers still, it is true, but barriers that could in time be swept away.”

Re: There are no new ideas in AI, only new datasets

#273

Post by tough »

IshKebab wrote: post #243
alternatex wrote: post #222

Earlier quoted context omitted.

Depending on the interviewer, you could make a non-AI program pass the Turing test. It's quite a meaningless exercise.

Obviously I mean for a sophisticated interviewer. Not nonsense like the Loebner prize.

The Turing test is contrived to chatting via textual interface.

These machines are only able to output text.

It seems hard to think they could reasonably think any -normal- person.

Tech only feels like magic if you don't know how it works

Re: There are no new ideas in AI, only new datasets

#274

Post by mikewarot »

delifue wrote: post #189
strangescript wrote: post #91

If you work with model architecture and read papers, how could not know there are a flood of new ideas? Only few yield interesting results though. I kind of wonder if libraries like pytorch have hurt experimental development. So many basic concepts no one thinks about anymore because they just use the out of the box solutions. And maybe those solutions are great and those parts are "solved", but I am not sure. How ma…

The hardware(GPU)'s architectural limitations may slow research more than PyTorch. The hardware lottery https://hardwarelottery.github.io/

I've been trying to get BitGrid [1] a hardware lottery ticket for a decade now.

[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...

Re: There are no new ideas in AI, only new datasets

#275

Post by swee69 »

nomel wrote: post #80
chasd00 wrote: post #52

Earlier quoted context omitted.

> Language and vision are just the beginning.. Based on the architectures we have they may also be the ending. There’s been a lot of news in the past couple years about LLMs but has there been any breakthroughs making headlines anywhere else in AI?

Two Nobel prizes in chemistry: https://www.nature.com/articles/s41746-024-01345-9

prizes != breakthroughs

Re: There are no new ideas in AI, only new datasets

#277

Post by sigmoid10 »

mym1990 wrote: post #198
sigmoid10 wrote: post #93

Earlier quoted context omitted.

Veo3's world model is still pretty limited. That becomes obvious very fast once you prompt out of distribution video content (i.e. stuff that you are unlikely to find on youtube). It's extremely good at creating photorealistic surfaces and lighting. It even has some reasonably solid understanding of fluid dynamics for simulating water. But for complex human behaviour (in particular certain motions) it simply lacks th…

What is the basis for it having a reasonable understanding of fluid dynamics? Why don’t you think it’s just regurgitating some water scenes derived from its training data, rather than generating actual fluid dynamics?

Because it can actually extrapolate to unseen cases while maintaining realism.

Re: There are no new ideas in AI, only new datasets

#278

Post by dinfinity »

yencabulator wrote: post #272
dinfinity wrote: post #235

Earlier quoted context omitted.

> Language is literally an abstraction of sensory inputs and cognitive processes. Language can exist entirely independently from senses and cognition. It is an encoding of patterns in the world where the only thing that matters is if anybody or anything wielding it can map the encodings to and from the patterns they encode for (which is more of a sociological/synchronisation challenge). Does C, or Java, 'make no sens…

> Language can exist entirely independently from senses and cognition. Helen Keller begs to disagree. Language and cognition were clearly linked for her. > It wasn't until April 5, 1887, when Anne took Helen to an old pump house, that Helen finally understood that everything has a name. Sullivan put Helen’s hand under the stream and began spelling “w-a-t-e-r” into her palm, first slowly, then more quickly. > Keller l…

I said language can exist independently, not that all language exists independently.

"one plus one equals two" can be understood and worked with without ever feeling water over your hand. It is a priori knowledge (see Hume's fork for an explanation).

You have to understand that the richness of language linked to cognition is due to your experience with that part of language and resulting romantization of it. It doesn't mean that it is a core defining feature of language, even though it feels that way (and as touching as that anecdote is).

Re: There are no new ideas in AI, only new datasets

#280

Post by actionfromafar »

dinfinity wrote: post #270
actionfromafar wrote: post #247

Earlier quoted context omitted.

Regarding visual communication, I think you downsell posturing, gesturing and facial expressions a little. They may not be as high bandwidth as talking but they are very low latency and pretty stealthy if necessary.

Well, there is sign language, so I guess you're right. It would be interesting to see how high bandwidth gesturing can be compared to speaking. I thought about this some more and I think the prevalence of making sounds rather than gesturing etc. is due to sound being a broadcasting mechanism that works over long distances and without line of sight. Visually indicating that you've claimed some territory is pretty hard…

I was thinking more about "low level" communication which can be gleaned from body language, frowns, smiles, gaze, winks, pointing etc. Perhaps not very information dense, but very fast.
Post reply on HN