It's so confusing because people keep (intentionally?) conflating two separate ideas of "AI safety". The first is the kind of humdrum ChatGPT safety of, don't swear, don't be sexually explicit, don't provide instructions on how to commit crimes, don't reproduce copyrighted materials, etc. Or preventing self-driving cars from harming pedestrians. This stuff is important but also pretty boring, and by all indications c…
> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like sayi…
I wouldn't be surprised by AGI tomorrow, which means I have to consider the possibility.
We have, at this point, an extremely good idea of how matter and energy behave. We also know roughly how long it takes to build physical objects and to do R&D cycles with physical prototypes in the loop and how much progress you get out of each prototype cycle. We have a number of good ideas for what a ray gun would look like and we know where we need advances. We've probably already made all of the necessary fundamental breakthroughs, things like laser diodes and capacitors and optical sapphire. We even have some deployed, just at artillery size and less-lethal energies rather than handheld blaster power densities. If you asked DARPA how long it would take them to field a ray gun for the US Army they could give you a date. So I would be extremely surprised if deployment was announced today. I would go looking for fundamental breakthroughs in battery tech and laser media that I had missed.
We can't even agree what AGI is, even on an "I'll know it when I see it" level. We have no idea how to get there. R&D cycles are days or weeks long and produce unpredictable and sometimes significant advances in observed functionality. Fundamental breakthroughs come unpredictably but probably about once every two to five years and their effects are even more unpredictable than the effects of individual R&D cycles. Even worse, we have no idea what required breakthroughs are left or if there even are any. So, yes, I believe that I would not be surprised by - and therefore must consider a future containing - one of these AI researchers glueing the output into the input in exactly the right way or something and producing a living, thinking being in their computer tomorrow.
I cannot concretely refute the possibility of AGI Tomorrow the way I can argue, in specific detail, against Ray Guns Tomorrow.
(There is, of course, the possibility that I've missed some breakthrough that would enable Ray Guns Tomorrow, the same way the people who pooh-poohed heavier-than-air flight the week before the Wright Brothers' fight missed gasoline engines and aerofoils and did their calculations based on a dude flapping wings with his arms. This is why I say that I would be extremely surprised, not why it's impossible.)