The Wait Equation and AI Investment
javednissar.ca
The Wait Equation and AI Investment
1–10 of 37 posts
Re: The Wait Equation and AI Investment
#2Reminds me of the classic paper "The Effects of Moore's Law and Slacking on Large Computations": https://arxiv.org/abs/astro-ph/9912202
Re: The Wait Equation and AI Investment
#3This completely misses the concept that the technology doesn’t just create itself without investment. SOMEONE has to invest for that progress to be made. It’s why I always hated the application of this to tech.
Re: The Wait Equation and AI Investment
#4Makes me think of this xkcd: https://xkcd.com/989/
Re: The Wait Equation and AI Investment
#5This completely misses the concept that the technology doesn’t just create itself without investment. SOMEONE has to invest for that progress to be made. It’s why I always hated the application of this to tech.
Investment is not the sole input into creating technology, time matters to. The tranfsormer paper probably didn't take much investment to create, but without it, any attempts at creating LLMs would have been futile no matter the billions poured in.
Moreover, technology is created as spillover effect from investments in other areas (That aren't speculative). CUDA is only possible because Nvidia was supported by gaming revenue for a decade. Without CUDA, there is no alexnet, and no deep learning boom will happen if we are still on CPUs or google's proprietary TPUs.
Re: The Wait Equation and AI Investment
#6Most AI investment is going to be applying off the shelf stuff to a new industry, so you might as well get started now to get the customers then when your wholesaler (OpenAI, AWS etc.) stocks the latest product you can wrap it up in a nice bow and sell it to your customers.
AI investment in SOTA though I have no idea how you time that but I am guessing early is better.
Re: The Wait Equation and AI Investment
#7This completely misses the concept that the technology doesn’t just create itself without investment. SOMEONE has to invest for that progress to be made. It’s why I always hated the application of this to tech.
Investment is not the sole input into creating technology, time matters to. The tranfsormer paper probably didn't take much investment to create, but without it, any attempts at creating LLMs would have been futile no matter the billions poured in. Moreover, technology is created as spillover effect from investments in other areas (That aren't speculative). CUDA is only possible because Nvidia was supported by gaming…
I’m sorry … but what?! As someone who worked on early
LLM tech at Google hundreds of millions were poured into it. Do you think that paper just spontaneously came from pure theory?
Re: The Wait Equation and AI Investment
#8This completely misses the concept that the technology doesn’t just create itself without investment. SOMEONE has to invest for that progress to be made. It’s why I always hated the application of this to tech.
> SOMEONE has to invest for that progress to be made.
History often has cases where the surrounding context/infrastructure/market/culture just wasn't suitable for an invention to take off--so it fails--and then somebody 5-100 years later tries something almost the same thing and it becomes a wild success and they get all the credit for it.
For example, look at the AT&T Picturephone from ~1964. [0] Even if someone from today time-traveled with the benefit of ~60 years of economic and technological hindsight... do you think a strategy of "just keep investing--somebody has to" would been enough to make the video-phone a widespread and profitable thing in any reasonable timeframe?
I'm pretty sure you'd go quite bankrupt first, just trying to stay solvent while making hundreds of other enabling-inventions and cost-savings in a wide range of fields while completely replacing global infrastructure.
Re: The Wait Equation and AI Investment
#9Earlier quoted context omitted.
Investment is not the sole input into creating technology, time matters to. The tranfsormer paper probably didn't take much investment to create, but without it, any attempts at creating LLMs would have been futile no matter the billions poured in. Moreover, technology is created as spillover effect from investments in other areas (That aren't speculative). CUDA is only possible because Nvidia was supported by gaming…
I’m sorry … but what?! As someone who worked on early LLM tech at Google hundreds of millions were poured into it. Do you think that paper just spontaneously came from pure theory?
Most of the core idea of transformers was invented in the early 1990s, including what at the time were termed Fast Weight Programmers and are formally equivalent to linearised self-attention: https://people.idsia.ch/~juergen/fast-weight-programmer-1991... . Google just had the hardware to actually run and experiment with large transformers.
Re: The Wait Equation and AI Investment
#10I think with AI it's a bit of a different question then space travel. The flip side you have a limited time to add value before the activity is no longer valuable due to AI.
In general once a machine can do something (and AI is a machine) it quickly becomes no longer a highly valuable activity.
For example, being a portrait painter around the time cameras were invented.
So if there's a project that you think AI will do for you, just keep in mind that by the time AI gets there the effective value of that activity will have greatly diminished and you are unlikely to get out of it what you would if you do it today.
If you wait, you may find you've been reduced from highly trained master of fine art to just another guy who says "say cheese" and pushes a button.
The value of problems is not fixed.
The space version is an interesting comparison though, because while the value of space exploration would increase with speed of travel (due to being able to make use of resources across greater distances), the value of any technological accomplishment decreases as they become easier.