Imagine if the original moores law tracked how often CPUs doubled the semi conductors while still functioning properly 50% of the time.
I don’t think it would have had the same impact
There are no new ideas in AI, only new datasets
241–250 of 307 posts
Re: There are no new ideas in AI, only new datasets
#242What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…
I don't get why people are so invested in framing it this way. I'm sure there are ways to do the stated objective. John Carmack isn't even an AI guy why is he suddenly the standard.
Maybe cause he's like top 5 most influential computer programmers of all time and knew to be a super human workaholic?
Re: There are no new ideas in AI, only new datasets
#243Earlier quoted context omitted.
> To humans this seems like they're truly solving novel problems Because they are . This is some crazy semantic denial. I should stop engaging with this nonsense. We have AI that is kind of close to passing the Turing test and people still say it's not intelligent...
Depending on the interviewer, you could make a non-AI program pass the Turing test. It's quite a meaningless exercise.
Obviously I mean for a sophisticated interviewer. Not nonsense like the Loebner prize.
Re: There are no new ideas in AI, only new datasets
#244Re: There are no new ideas in AI, only new datasets
#245Earlier quoted context omitted.
Yes and: and forget.
Why is forgetting important? Things can either have an end time where they are no longer applicable or things we thought were true turn out to be false but it's still useful to see where we went wrong. I imagine humans are limited by the # of synapses we have so it's useful to forget but maybe machines can move the useless stuff to deep storage until it's dug out, in the same way certain things can trigger a deep mem…
Which is cheaper, among these two logically equivalent things: reducing one weight, or increasing every other weight?
Re: There are no new ideas in AI, only new datasets
#246I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…
I think one counterpoint to this idea is the compute cost. To a great extent, it's not AI research that is the primary driver behind the huge advances in AI, either in terms of techniques (transformers) or data sets. Instead, the biggest single factor responsible for this huge boost are advances in compute hardware and compute power in general. Even if we had known about the Transformer architecture 20 years earlier,…
Tick-tock model. Research switches between adding new ideas and pouring more power into good ideas.
Re: There are no new ideas in AI, only new datasets
#247Earlier quoted context omitted.
> Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence. I probably made a mistake when i asserted that -- should have thought it over. Vision is evolutionarily older and more “primitive”, while language is uniquely human [or maybe, more broadly, primate, cetacean, cephalopod, avian...] symbolic, and abstract — arguably a different ord…
Vision is interesting in that it leverages the maximum speed with which it is easily possible to gather information about our surroundings in this universe. I believe that is what makes it special and very valuable. I also believe this aspect makes it a strong attractor for convergent evolution. Language allows encoding and compression of information about the world, which is of course incredibly powerful and increas…
Regarding visual communication, I think you downsell posturing, gesturing and facial expressions a little. They may not be as high bandwidth as talking but they are very low latency and pretty stealthy if necessary.
Re: There are no new ideas in AI, only new datasets
#248Earlier quoted context omitted.
People who lack one of those senses, or even two of them, tend to do just fine.
Mostly thanks to other humans helping them. If all humans lacked vision, the human race would definitely not do just fine.
I think we need to think about vision and world modelling somewhat separately. We could construct an artificial (tech enhanced) society where sight was not available. People would still "model the world in their minds" with the "abstract model" part of the vision/world system.
Re: There are no new ideas in AI, only new datasets
#249Earlier quoted context omitted.
> You can trivially prove this yourself. Given the long list of dead philosophers of mind, if you have a trivial proof, would you mind providing a link?
Just go and ask ChatGPT or Claude something that can't possibly be in its training set. Make something up. If it is only memorising answers then it will be impossible for it to get the correct result. A simple nonsense programming task would suffice. For example "write a Python function to erase every character from a string unless either of its adjacent characters are also adjacent to it in the alphabet. The string…
[deleted]
Re: There are no new ideas in AI, only new datasets
#250I'd say with confidence: we're living in the early days. AI has made jaw-dropping progress in two major domains: language and vision. With large language models (LLMs) like GPT-4 and Claude, and vision models like CLIP and DALL·E, we've seen machines that can generate poetry, write code, describe photos, and even hold eerily humanlike conversations. But as impressive as this is, it’s easy to lose sight of the bigger…
> Language and vision are just the beginning — the parts we were able to digitize first - not necessarily the most central to intelligence. I respectfully disagree. Touch gives pretty cool skills, but language, video and audio are all that are needed for all online interactions . We use touch for typing and pointing, but that is only because we don't have a more efficient and effective interface. Now I'm not saying t…
Humans are known for their exceptional sensitivity in their hands and fingers. There are only few animals that come close to our ability to manipulate objects.
Only octopuses, elephants and apes are in a similar league with regards to dexterity and finesse.