161–170 of 267 posts
I'm sure the government's mission is also to develop an AGI that benefits us all.
Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.
If a company has it, US intelligence will have it, for sure.
If US govt has it, it might take a decade, or so, to leak out.
Earlier quoted context omitted.
> I bet if you add up that data it's a lot. Let's Fermi estimate that. A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25. Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.…
So much of that data is totally useless though. Most of the data we get is visual and I would argue that visual data is some of the least useful data (with respect to volume). Think about the amount of time we're looking at blank colours with nothing to learn from. Once you've seen one wall, one chair, one table etc, there's not much left to know about those things. An encyclopedia though for example is much less dat…
Earlier quoted context omitted.
Ideally no one person or entity controls such a thing. But, would I rather have a Government, or a corporation control AGI? If I had to pick one of two evils, the Government would be the lesser of the two.
Yeah but AGI would control both in the end.
Earlier quoted context omitted.
Then the first thing the AGI will do is figure out ultra-high throughput analog computing for LLMs
An AGI will likely be scared of even more powerful AGI's that it itself cannot control. It might take steps to try to prevent anyone building such a thing.
Earlier quoted context omitted.
This is a particular interesting part of American culture: The sentiment is that it is problematic when the government develops these technologies, but it is completely OK to let private entities develop them.
There's only one US Government and we know it did fairly bad things. There are many private entities, although often not benevolent, most still have space for the benefit of doubt. OpenAI, DeepMind or Anthropic haven't done anything that compares to, say, MKUltra or lying about Iraqi WMDs to start decade long wars.
There is no line between government and corporations. These are all groups of people, and some people do bad things. The only way to prevent bad things is to do them in the open and have oversight, and that’s more likely to happen in the government than in private companies, maybe.
Earlier quoted context omitted.
Was it ever confirmed whether GPT-4 is a Mixture of Experts or not?
I just asked GPT-4, and it denies being a MoE model. User: Are you an MoE model? ChatGPT: No, I am not based on a Mixture of Experts (MoE) model. My underlying architecture is based on the GPT (Generative Pre-trained Transformer) framework, specifically the GPT-4 version. This architecture is a large-scale transformer-based neural network, but it does not use the MoE approach. In a GPT model like mine, the entire mod…
Earlier quoted context omitted.
Yeah I'm not totally convinced humans don't have a tremendous amount of training data - interacting with the world for years with constant input from all our senses and parental corrections. I bet if you add up that data it's a lot. But once we are partially trained, training more requires a lot less.
> I bet if you add up that data it's a lot. Let's Fermi estimate that. A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25. Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.…
Earlier quoted context omitted.
Such as when energy companies buried research about climate change...?
Then again it was the Greens that blocked nuclear power. In Germany, their actions actually caused increase in the use of coal and increased CO2 output.
So it’s possible for a rational environmentalist to acknowledge the benefits and drawbacks of nuclear and decide subsidizing nuclear is a poor investment vs other options. Since 2010 Germany reduced coal production by 82 TWh while also reducing Nuclear by 104 TWh. Oil -7 TWh and gas + 6TWh effectively canceled out for a net of -83 TWh from fossil fuels.
The solution won't be just "bigger". A model with a trillion parameters will be more expensive to train and to run, but is unlikely to be better. Think of the early days of flight, you had biplanes; then you had triplanes. You could have followed that farther, and added more wings - but it wouldn't have improved things. Improving AI will involve architectural changes. No human requires the amount of training data we…
It does seem like bigger models give better responses when given benchmarks. It might plateau or overfit the data at some point, but I'm not sure we've reached it yet.
I'm sure the government's mission is also to develop an AGI that benefits us all.
I think AI could be turned into a very effective persuasion weapon. I think the government that does it first will gain more power as a result.