> Megabyte
> not Metabyte
You had one job...
Meta AI Unleashes Megabyte, a Scalable Model Architecture
171–180 of 213 posts
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#172Earlier quoted context omitted.
When I started reading this and immediately came across the word "groundbreaking" I paused for a sec and thought to myself: Let me just pretend this word isn't there, it's probably an exaggeration. It takes a special kind of adaptation to filter out all the 10%-40% bluff usually found in these kinds of news articles.
Now I kind of want to see a browser extension that deletes all the intensifier adjectives from runs of text, and is able to be enabled/disabled per domain.
So a less unhinged version of [0]?
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#173Earlier quoted context omitted.
This. So much this. I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go. Like anyone would expect a human programmer or researcher to just intuitively come up with a complex program, or the correct answer for a hard problem every time…
> I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go. People are very comfortable with siloed information, even smart people. This is why we have 100 different words for the same concept across different areas of science, industry an…
NOVA has an amazing episode on how completely deluded we are at how our brain actually works. Our consciousness spends a lot of time lying to us.
https://www.pbs.org/video/your-brain-perception-deception-pr...
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#174Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#175Earlier quoted context omitted.
When I started reading this and immediately came across the word "groundbreaking" I paused for a sec and thought to myself: Let me just pretend this word isn't there, it's probably an exaggeration. It takes a special kind of adaptation to filter out all the 10%-40% bluff usually found in these kinds of news articles.
Now I kind of want to see a browser extension that deletes all the intensifier adjectives from runs of text, and is able to be enabled/disabled per domain.
Bit random, but a while ago I created this extension to remove "actually", "literally" and other words (it is slighty configurable) that can be annoying. It is sort of a joke, but some days I think it is not a joke.
https://chrome.google.com/webstore/detail/actually-no-remove...
Maybe I should extend it :)
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#176Earlier quoted context omitted.
Your premise is wrong. This has nothing to do with 10x-ing parameters. One could argue the current parameter sizes are good enough as we observe "large breadth, shallow depth" behavior from LLM and to some extent, diffusion models. This suggests the problem is the depth of inference, which is single pass "hot takes" for all language models right now, due to cost of inference and our limited understanding of what make…
This. So much this. I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go. Like anyone would expect a human programmer or researcher to just intuitively come up with a complex program, or the correct answer for a hard problem every time…
> I've been actively looking for good arguments against AGI being much closer than we should be comfortable with, and reasons why we should not fear systems that surpass us in intelligence.
> My guess is these people will act surprised when the arms race inevitably leads to some very bad unintended consequences.
One argument to keep in mind is that if you take a pessimistic view then you will eventually be right. If you predict the current LLMs will eventually be involved in some bad thing then you might even feel self-satisfied when a different bad thing happens as if you predicted the specific way in which it caused the problem.
What I mean to say is, it seems unlikely that paper-clip maximizers will be our undoing. But just vaguely gesturing and saying "something bad will probably happen" isn't as useful as we would like to think. And even enumerating the 100s of possible ways something might go wrong has a diminishing returns kind of quality to it. It's like a hypochondriac insisting he has every disease known to man and then exclaiming "I told you so!" when a doctor diagnoses him with a cold.
If you venture into that vague kind of "I have a bad feeling about this AI stuff" territory, you are on no more (or less) solid ground than the AI hype evangelists. While I don't want to go all Oprah and "The Secret" or some law of attraction pseudo-rationality ... I feel it is worthwhile focusing a little more on the possible benefits rather than allow ourselves to be swayed by vague fears of potential disasters.
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#177Paper: https://arxiv.org/abs/2305.07185 Wow, seems like Meta AI is so ahead of the curve compared to even Google and OpenAI recently especially with their open sourcing pushes. Great for the research community as a whole!
Meta has no horse in the race (i.e. they don't have a search engine). So, they don't mind throwing random things out. Withholding it won't really make much of a difference for them, as they don't have a way to productionize the tech.
Meta still has the Portal line of products - which were competitors to the Alexa, Siri and similar Google audio assistant product lines. I just searched and it looks like Meta are currently partnering with Amazon to license Alexa on these devices, but I could imagine they might want to replace it with their own LLM eventually.
I am a bit surprised that no one is talking about how these new LLM models will disrupt Alexa, Siri, etc. since that seems to be the most applicable market I can imagine.
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#178Earlier quoted context omitted.
I consider the singularity to be the point at which the certainty of our predictions about our future becomes close to zero. By this definition I reckon we are already in the singularity. It’s not necessarily bad. The problem with the singularity is that that we can’t tell if it’s bad or not.
It's hard to predict some things now, but the vast majority of things are as easy to predict as they've ever been, or we wouldn't be reasonably sure that when we took a step on the sidewalk that our feet would touch the ground, or that when we opened a door leading in to a house there'd be a house on the other side, or that cars, airplanes, and other machines mostly work as designed, that food is nutritious and satia…
>or we wouldn't be able to survive.
If you want to keep surviving you have to be 100% correct in your ability to predict the correct choice. Now the thing is the vast majority of predictions you have to make about anything are not necessarily life or death and/or rapidly changing. LIke you said, cars are still cars, and food is still food.
The problem is the change can be massively abrupt. Watching the use of drones in Ukraine is seemingly a decent example of this. Small cheap drones are being used in mass as surveillance platforms, for artillery targeting, and even for direct enemy bombing. They cost of defending against them seems to be far higher than the cost of using them to attack.
Now this isn't exactly unpredictable in itself. Where is gets more questionable about predictions, is what is it going to look like when AI is given direct control of swarms of these things rather than individuals with an xbox controller? Suddenly decades of battle field planning is out the window and there is a massive shakeup in fighting capabilities.
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#179Earlier quoted context omitted.
> How do you think a human with below average intelligence (or even with average intelligence) would fare? I don't understand what rote memorization to pass a test has to do with intelligence. For the record Kim Kardashian passed the bar; I imagine anyone given the proper motivation and time to study could do it, it's not a hard test. If AGI is a computer passing a test, then AGI was achieved a long time ago. I don't…
> If AGI is a computer passing a test, then AGI was achieved a long time ago. AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing. > it's clear LLMs are not AGIs And the main argument for that is that "it's clear". They're beating lawyers, doctors, and software engineers, but obviously, that's not real inte…
>> If AGI is a computer passing a test, then AGI was achieved a long time ago.
> AIs couldn't even pass a third-grade reading comprehension exam until about 5 years ago. Computers being able to pass tests designed for humans is a very new thing.
You're narrowly defining what a "test" is. And therefore the "until 5 years ago" doesn't make any sense. A test is not just written exams like a "third-grade reading comprehension exam." Is playing chess against the world champion not a test? A trad AI computer program beat him 25 years ago, not 5 years ago. Is diagnosing patients with a particular disease better than the average untrained human not a test? An expert system did that 50 years ago.
Your point is not a refutation of the parent post.
Re: Meta AI Unleashes Megabyte, a Scalable Model Architecture
#180Earlier quoted context omitted.
This. So much this. I'm completely dumbfounded by obviously highly intelligent people consistently not getting this, and dismissing current generation AI systems as not being intelligent because they can't reliably solve massively complex problems in one go. Like anyone would expect a human programmer or researcher to just intuitively come up with a complex program, or the correct answer for a hard problem every time…
I would add to your amazing list that we are really good at denial as a coping mechanism with change. I am not a fan of the concept of AGI though. This means so many different things to people that it seems pointless to debate something when most likely we are not talking about the same thing. François Chollet has said that he believes all intelligence is specialized intelligence. From that perspective, whatever peop…
> The doomer argument though is coming from defending our highly affluent and privileged life
It's not at all about that.
Even if "truly general" intelligence is impossible, that's irrelevant to the actual concerns about AI apocalypse. There are multiple theories about what failure looks like, but they essentially come down to a loss of control.
Now, obviously, that means something different for the owner class and for the worker class, which can be extrapolated to have global implications as well. But this isn't an issue of the owner class ceding control to the working class. It's an issue of the owner class ceding control to an alien. Maybe that alien makes things more egalitarian and prosperous. Or maybe it makes us extinct. Any and all possibilities are options for it as far as we know because it is fundamentally an inhuman (= alien) intelligence. We can't understand it even as well as we understand humans and human organizations (that is, not very well), let alone control it as well as we do humans and human organizations (that is, not enough to prevent self-inflicted climate apocalypse).
Basically, we're opening a box with a random magical spell inside it and deciding that we'll just have to live with whatever the effects of that spell are. I'm not for the status quo, but AI is just mind-bogglingly dangerous, and I think that's why there are so many wrong arguments against its danger. We literally cannot comprehend an intelligence greater than our own.