I can imagine AI academics are in a tough spot. However what about the existential angst the rest of us - who don't even do anything AI related on a day to day basis - are feeling? I think big changes are coming and it's not gonna be pretty.
Choose Your Weapon: Survival Strategies for Depressed AI Academics
21–30 of 195 posts
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#22Earlier quoted context omitted.
I don't think they are putting themselves on a pedestal. It's more that they made the choice to remain in academia, because they believed that were trading a higher salary for the academic freedom and the chance to work with really cool problems. And then it turns out, industry are the ones really working with the really cool problems. (Disregarding the fact that ~85% of machine learning academics do not have what it…
As someone in this space I can attest that AI teaching in most (UK) universities is generally poor on detail, abstract and behind industry by at least 3-5 years. Not to mention that there is zero appetite from undergrads or postgrads to get into the nitty-gritty of it. To learn CNNs at the deep-dive level you need calculus, at least differentiation and integration. Calculus or even pre-calculus doesn't form part of t…
lmao even in my university (the serbian uni), we have at least calculus+linear algebra before any nn course. also to "learn" what's a cnn you just need gradients not integrals (unless you use some kind of non-lipschitz function as activation?), plus the idea of what a convolution is... but even Mobius knew it back in the 800s.
anyway i think your statement that industry is light years away from unis is just misleading. i think the two are trying to answer different questions: 1. how can i achieve a "somewhat" decent chatbot that gets me rich albeit not even knowing what it does [industry in case you wondered] 2. try to understand, quantify and measure how well a model works, is it stable? does it converge if we have small datasets? and so on so forth.
just my two cents, to conclude i think a good analogy to the current climate is the 700-800s with electromagnetism: plenty of people discovered "empirical" laws but didn't understand really the phenomenon.
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#23[flagged]
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#24Obviously AI academics are uniquely challenged by the 'scaling is (nearly) everything' reality, but I feel like a lot of the mixed emotions being expressed towards the latest results are also because we're actually seeing the mystery of self starting to unravel. Like any good mystery, the fun was in the build up and as we move towards a resolution there's a bitter sweet aspect to the slightly mundane reality of it 'm…
Nothing about current AI models profoundly challenges the unknowns of what makes consciousness work in any mystery-resolving way. People who confuse plainly programmed algorithmic systems like GPT 4 or Midjourney with the still unresolved issues of sentience as we know it so far are drinking far too much of the current batch of AI punch. Sadly, it's a sentiment I see all too often on HN, a site in which i'd assume most readers and commentators would be a bit more skeptical-minded about these things. It's almost funny, considering how much hate this site tends to throw at things like crypto, to see so much of the wide-eyed opposite with the latest AI craze.
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#25Obviously AI academics are uniquely challenged by the 'scaling is (nearly) everything' reality, but I feel like a lot of the mixed emotions being expressed towards the latest results are also because we're actually seeing the mystery of self starting to unravel. Like any good mystery, the fun was in the build up and as we move towards a resolution there's a bitter sweet aspect to the slightly mundane reality of it 'm…
> So now the sadness comes. The revelation. There is a depression after an answer is given. It was almost fun not knowing. Yes, now we know. At least we know what we sought in the beginning. But there is still the question, why? And this question will go on and on until the final answer comes. Then the knowing is so full there is no room for questions.
- Twin Peaks, 1993
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#26This papar is really interesting, especially so when you scroll to the Reference part ;)
Unfortunately, having checked Ref 7, I now can't unsee Chuck Tingle.
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#27Obviously AI academics are uniquely challenged by the 'scaling is (nearly) everything' reality, but I feel like a lot of the mixed emotions being expressed towards the latest results are also because we're actually seeing the mystery of self starting to unravel. Like any good mystery, the fun was in the build up and as we move towards a resolution there's a bitter sweet aspect to the slightly mundane reality of it 'm…
Nothing about current AI models profoundly challenges the unknowns of what makes consciousness work in any mystery-resolving way. People who confuse plainly programmed algorithmic systems like GPT 4 or Midjourney with the still unresolved issues of sentience as we know it so far are drinking far too much of the current batch of AI punch. Sadly, it's a sentiment I see all too often on HN, a site in which i'd assume mo…
Yeah, it reads so bizarrely to me... How are we confusing biological human (or other animal) sentience in some embodied creature with queries as a service to some model instance.
I cannot see anything that remotely makes these two contexts architecturally, practically, or ethically similar, except in the narrow sense of query responses that might have been mistakenly thought of as defining sentience in earlier decades.
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#28Earlier quoted context omitted.
It may or may not. I mean it’d be pretty hard to increase yields of GPU fabs or data center sizes another 100x. There are logistical limitations. Unless some Apollo level mission is created by a superpower, we will hit bottlenecks. Algorithmic innovation is the only long term bet.
I thought that too, a year ago. But then chatGPT Turbo came out (ten times cheaper), and a slew of 30B and 10B models that are decent. Now I believe we will be able to run non-trivial AI on trivial hardware. Not to mention the Stable Diffusion revolution, hardware requirements went down pretty fast. Even an old GPU from 5 years ago can generate images quickly. Some LLMs run on iPhones. The trick is always to offset t…
And? ChatGPT doesn't give you logprobs. It doesn't allow you to alter the generation algorithm. They've left their lunch out on the table for someone else to eat.
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#29Academics can’t put themselves on a pedestal for being revolutionaries who care about the ultimate truth the most and at the same time whine when an advance comes along. These CS academics need to understand that this is what it feels like to be in other fields like physics or bio where you can’t do jack unless you have costly equipment. This is what people in developing countries deal with all the time. And people w…
> These CS academics need to understand that this is what it feels like to be in other fields like physics or bio where you can’t do jack unless you have costly equipment. This is what people in developing countries deal with all the time. This was my thought too. It cost approximately $4.75 billion to build the Large Hadron Collider at CERN. My educational background is in physics, but I would not be the least bit u…
I don't know, core physics leading to Fusion reactors and FTL travel seen much more important to me than AI that can make convincing robocalls and generate furry porn stories.
Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics
#30Earlier quoted context omitted.
I don't think they are putting themselves on a pedestal. It's more that they made the choice to remain in academia, because they believed that were trading a higher salary for the academic freedom and the chance to work with really cool problems. And then it turns out, industry are the ones really working with the really cool problems. (Disregarding the fact that ~85% of machine learning academics do not have what it…
> Disregarding the fact that ~85% of machine learning academics do not have what it takes to be hired by OpenAI anyway That's interesting, could you please elaborate?
It's not interesting. You can regard OpenAI as one of the most prestigious AI labs in the world and only the best of the best get a chance to work there.