21–30 of 37 posts
I sometimes think of a bigger version of this question. Would it be better if rennaisance and scientific revolution had come way later than it did. Maybe humans would have been more evolved and ready for the changes and can make better decisions. Progress is inevitable, but the rapidity of change might be our downfall.
Earlier quoted context omitted.
I’m sorry … but what?! As someone who worked on early LLM tech at Google hundreds of millions were poured into it. Do you think that paper just spontaneously came from pure theory?
Most of the core idea of transformers was invented in the early 1990s, including what at the time were termed Fast Weight Programmers and are formally equivalent to linearised self-attention: https://people.idsia.ch/~juergen/fast-weight-programmer-1991... . Google just had the hardware to actually run and experiment with large transformers.
Someone pointed out that they would have been better off waiting for about 40 years of Moore's law to happen, then building a computer and running the same calculation in about 2 years.
I think this analogy is best suited for training models. Even if you had access to OpenAIs datasets right now, I don't think it would make sense for you to train them, except if you are a 500B+ company. Training costs will likely go down with time though, so at some point this might change.
This completely misses the concept that the technology doesn’t just create itself without investment. SOMEONE has to invest for that progress to be made. It’s why I always hated the application of this to tech.
Earlier quoted context omitted.
I’m sorry … but what?! As someone who worked on early LLM tech at Google hundreds of millions were poured into it. Do you think that paper just spontaneously came from pure theory?
Most of the core idea of transformers was invented in the early 1990s, including what at the time were termed Fast Weight Programmers and are formally equivalent to linearised self-attention: https://people.idsia.ch/~juergen/fast-weight-programmer-1991... . Google just had the hardware to actually run and experiment with large transformers.
The wait equation is totally irrelevant here. Where the wait equation has relevance is the fact that a generation ship, once launched, is likely unable to reconfigure itself to take advantage of new technologies invented while it is underway. This is key to why there is an incentive to wait, and there is no parallel in the idea of investing in AI. Someone who chooses to invest in AI does get access to tech that devel…
Given my experience with complex technology (software), I think it's rather likely that a generation ship will reconfigure while traveling.
Maybe we will build a ship that's just enough to bring them to a place with lots of resources. Then, it will be rebuild with those resources for the final journey. Or even new ships might be build out of existing ones to adapt for varying conditions.
Similar to biological life on earth.
Earlier quoted context omitted.
But we did people continuously invested through the 80s then 90s then Cisco then Google and now zoom. Do you and the writer not know that video calling was a consistent investment for over 40+ years?!
By that logic, somebody was so desperate to get convenient warm food that they decided to "invest" for decades in side-projects like airplanes and bombs just to finally get a microwave-oven, obviously true purpose of everything that came before. Just because something else in civilization might somehow help someday isn't enough to make it part of "the investment" which a company or individual is deciding whether to m…
The wait equation is totally irrelevant here. Where the wait equation has relevance is the fact that a generation ship, once launched, is likely unable to reconfigure itself to take advantage of new technologies invented while it is underway. This is key to why there is an incentive to wait, and there is no parallel in the idea of investing in AI. Someone who chooses to invest in AI does get access to tech that devel…
> Where the wait equation has relevance is the fact that a generation ship, once launched, is likely unable to reconfigure itself to take advantage of new technologies invented while it is underway. Given my experience with complex technology (software), I think it's rather likely that a generation ship will reconfigure while traveling. Maybe we will build a ship that's just enough to bring them to a place with lots…
Pretty much all scenarios for accelerating to cruise speed and decelerating at the other end involve very high density fuels produced using a significant fraction of the energy of the Sun. You're not going to be able to stop in the middle of empty space where there is absolutely nothing in order to reconfigure and refuel.