31–40 of 92 posts
What I mean by that is that AIs, the way they are currently built, need to learn very slowly on short term inputs or they overfit. Whereas humans can learn something just by explanation short term and don't have overfitting problems.
I suspect this is solved by sleep, and I haven't seen AI with a similar mechanism.
I suspect biological brains have a pretty groundbreaking hack to solve the long-term short term learning problem. Maybe involving sleep. What I mean by that is that AIs, the way they are currently built, need to learn very slowly on short term inputs or they overfit. Whereas humans can learn something just by explanation short term and don't have overfitting problems. I suspect this is solved by sleep, and I haven't…
This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…
Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…
This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…
I don't think so. If you want to model a single synapse in full to capture all effects that might lead to "learning", you have a system of ordinary differential equations. Solving that is very hard, and solving that for 10 million neurons is impossible.
On current hardware can only implement but a poor caricature of a real neuron.
This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…
Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…
And it's understanding what the right thing is to build that's the critical challenge in programming.
As a theory person who usually explains O notation using concrete numbers, the degree of the neural network in our brain is approx 7000. Taking approx 86 billion ~ 100 billion, this itself is a graph with approx 6x10^(14) edges - does AGI proponents really hope to be able to do this? I am genuinely curious to know : is there some simplifying assumption which makes things faster?
That’s only 3 orders of magnitude off from today’s largest models like PaLM (5x10^11 parameters), a gap that’s narrowed by 3 orders of magnitude just since 2019. How far away do you think we are, exactly?
PaLM is not an attempt at AGI, a parameter is not equivalent to a neural connection, an activation function is not equivalent to a neuron (of which you have many different types), biological connection patterns are much richer, and biological stimuli are not like slideshows of a single type of data, so...
I think this essay includes a specific prediction, that human level ai is far away, that might be disproved this decade. If human level ai is close, focusing on some other kind of ai is more likely to be a waste of time.
Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day.
https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...
Earlier quoted context omitted.
Imagine if the immediate outcome of AI is not that we replace taxi drivers, dishwashers, and factory workers, but instead we displace most knowledge-worker white collar jobs, like quant and software engineer? There's an old (and sometimes forgotten) idea in AI that perhaps things we think are simple, like vision and control (robotics), are actually incredibly complicated and took millions of years to evolve. Whereas…
I think there's a lot of truth to this. I'm new to ML, still going through the ropes on some online courses, but already I can see that, once I get a bit of muscle memory in setting up models etc, there's a whole lot of power and efficiency to be unlocked by using simple models - specifically in CS/X and Marketing. Obviously model quality matters, so you have to have proper monitoring etc, but this stuff is low hangi…
I've worked on a number of projects where it looked simple to automate from the outset and impossible in retrospect.
This is a common sentiment, and pundits have been making similar remarks for decades. This author writes "Sixty years later, however, high-level reasoning and thought remain elusive." That's the wrong problem with AI. The trouble with AI is that it still sucks at manipulation in unstructured situations and at "common sense". Common sense can usefully be defined as getting through the next 30 seconds of life without a…
One thing that has held back progress is the way putting knowledge directly into the system has become taboo. So much so that they often fail to even guide the training towards really core aspects of the world model. Or even deliberately going about it with the assumption that everything from start to finish must be determined from the barest input data such as pixels. Then being surprised when it learns random inaccurate and overfit models that miss the underlying hierarchical structures.
I suspect biological brains have a pretty groundbreaking hack to solve the long-term short term learning problem. Maybe involving sleep. What I mean by that is that AIs, the way they are currently built, need to learn very slowly on short term inputs or they overfit. Whereas humans can learn something just by explanation short term and don't have overfitting problems. I suspect this is solved by sleep, and I haven't…
How is sleep involved? There are a lot more differences than sleep.