Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Choose Your Weapon: Survival Strategies for Depressed AI Academics

arxiv.org

Post reply on HN
Print view View original

101–110 of 195 posts

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#101

Post by TeMPOraL »

dontupvoteme wrote: post #71
jksk61 wrote: post #22

Earlier quoted context omitted.

lmao even in my university (the serbian uni), we have at least calculus+linear algebra before any nn course. also to "learn" what's a cnn you just need gradients not integrals (unless you use some kind of non-lipschitz function as activation?), plus the idea of what a convolution is... but even Mobius knew it back in the 800s. anyway i think your statement that industry is light years away from unis is just misleadin…

>just my two cents, to conclude i think a good analogy to the current climate is the 700-800s with electromagnetism: plenty of people discovered "empirical" laws but didn't understand really the phenomenon. Sounds dead on. Do these large """language""" models actually even implement any concepts from linguistics? Or is the entire "language" part of the model merely derived from the fact that it's inherently part of t…

>> anyway i think your statement that industry is light years away from unis is just misleading. i think the two are trying to answer different questions: 1. how can i achieve a "somewhat" decent chatbot that gets me rich albeit not even knowing what it does [industry in case you wondered] 2. try to understand, quantify and measure how well a model works, is it stable? does it converge if we have small datasets? and so on so forth.

GP here is, IMO, confusing what the corporations want (1), with what corporate R&D people want (2). As long as the corps see good ROI on throwing infinite money at their AI R&D departments, then those corporate researchers are better positioned and better equipped to do actual, solid science, than academia ever can be. This has happened many times before, including in this industry. Research is best done by well-funded teams of smart people left to do whatever they fancy. When those conditions arise, progress happens, and it doesn't matter whether it's the government or industry that creates them.

(Conversely, the best hope for academia to become relevant again is that corporations lose interest in this research, and defund their departments. This could happen if e.g. transformers end up being a dead end, or compute suddenly becomes very expensive.)

> Do these large """language""" models actually even implement any concepts from linguistics? Or is the entire "language" part of the model merely derived from the fact that it's inherently part of the training data?

The latter. And guess what, they're not trying to solve the issue of linguistics. They started as tools to generate human-sounding text, but in the process of just throwing more data and compute at them, they not only got better, but started to acquire something resembling concept-level understanding.

It turns out that surprisingly many aspects of thinking seem to reduce well to proximity search in a vector space, if that space is high-dimensional enough. This result is both surprising and impactful well beyond the field of AI. It's arguably the first potential path we identified that the evolution could take to gradually random-walk itself from amoeabas to human brains.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#102

Post by johnaspden »

mr_gibbins wrote: post #19
numbers_guy wrote: post #17

Earlier quoted context omitted.

I don't think they are putting themselves on a pedestal. It's more that they made the choice to remain in academia, because they believed that were trading a higher salary for the academic freedom and the chance to work with really cool problems. And then it turns out, industry are the ones really working with the really cool problems. (Disregarding the fact that ~85% of machine learning academics do not have what it…

As someone in this space I can attest that AI teaching in most (UK) universities is generally poor on detail, abstract and behind industry by at least 3-5 years. Not to mention that there is zero appetite from undergrads or postgrads to get into the nitty-gritty of it. To learn CNNs at the deep-dive level you need calculus, at least differentiation and integration. Calculus or even pre-calculus doesn't form part of t…

> To learn CNNs at the deep-dive level you need calculus, at least differentiation and integration.

Oh the poor dears, imagine needing schoolboy maths to do science.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#103

Post by leroy-is-here »

elif wrote: post #92
leroy-is-here wrote: post #66

Earlier quoted context omitted.

I’m trying to show you something miraculous. It simply involves understanding that you are not your thoughts.

A simple linguistic ontology is one of the most perverse, heuristic patterns of thought. A rock is also more than it's thoughts, and retains some degree of consciousness. Imagining concepts like souls as a sort of litmus test for consciousness only serves to reinforce a thought-based comprehension.

I'm asking you to stop arguing inside "simple linguistic ontology".

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#104

Post by dmd »

sseagull wrote: post #97
CuriouslyC wrote: post #75

Earlier quoted context omitted.

What an opportunity! I get to spend most of my time writing grant proposals, compete with 30 other associate professors for 1 tenure track job, then when I get tired of that I can switch to a marginal wage industry job because AI researchers are the only scientists that get good compensation as a group, and the 28 other people who didn't get that tenure position are probably also trying to do the same thing (thus dep…

While true, there's more to academia that professors. There's a growing number of research scientists (particularly in software), as well as positions national labs (if you count that as pseudo-academia). There's also lots of room in applying AI techniques to other areas of science (again, outside of professorships)

Yep. But we get paid at academic rates. I stay for the relaxed lifestyle, not the salary.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#105

Post by TeMPOraL »

martopix wrote: post #98
gitfan86 wrote: post #54

Earlier quoted context omitted.

OpenAI wasn't created to "Do Science". It was created to give humanity a better chance at navigating the singularity.

> OpenAI wasn't created to "Do Science". That's exactly OP's point

OpenAI was created to solve a problem that involves doing science, so... at the moment, it absolutely does do science.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#106

Post by foooorsyth »

bo1024 wrote: post #52

As a theorist in AI, the premise is funny but also really sad. And yet it is a very pervasive premise in the field. OpenAI is not doing science. They are building a big shiny thing, showing it off, keeping it closed, and making money off of it. That is not part of the scientific process. It is as if, every time SpaceX launched a rocket a little bit higher, every aerospace department ooh'd and aah'd and threw up their…

>That is not part of the scientific process

What? OpenAI is absolutely conducting science amongst themselves. They are certainly engaging in the hypothesis -> test -> result scientific method, which is science in its purest form. Sharing the secret sauce is what is done or expected in academia, but that doesn’t mean that OpenAI isn’t conducting science behind their own walls. You’re conflating institutional academia with the scientific method.

If I go home and test the boiling point of water in my garage and record the result privately without publishing a paper, am I not conducting science? What nonsense.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#107

Post by ndriscoll »

jksk61 wrote: post #22
mr_gibbins wrote: post #19

Earlier quoted context omitted.

As someone in this space I can attest that AI teaching in most (UK) universities is generally poor on detail, abstract and behind industry by at least 3-5 years. Not to mention that there is zero appetite from undergrads or postgrads to get into the nitty-gritty of it. To learn CNNs at the deep-dive level you need calculus, at least differentiation and integration. Calculus or even pre-calculus doesn't form part of t…

lmao even in my university (the serbian uni), we have at least calculus+linear algebra before any nn course. also to "learn" what's a cnn you just need gradients not integrals (unless you use some kind of non-lipschitz function as activation?), plus the idea of what a convolution is... but even Mobius knew it back in the 800s. anyway i think your statement that industry is light years away from unis is just misleadin…

You might be able to understand what a convolutional network "is" without calculus, but you'll be woefully unequipped to ask even obvious questions like "what if we put Fourier transforms around the convolutional layers" (a cursory search suggests it provides the expected speedup but is for some reason not a standard thing to do?). As someone outside of the industry, I'd also imagine any effort to explain what NNs are actually "learning" (or I suppose dually, how to design network architectures) is going to have a lot of fruitful overlap with signal processing theory, which is heavy on calculus, linear algebra, probability, etc.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#108

Post by phkahler »

TeMPOraL wrote: post #89
matthewdgreen wrote: post #40

Earlier quoted context omitted.

I think your response is very narrow in defining what a "really cool problem" is. If in the 1970s you imagined that the cool problem in ECE/CS was "building ever-more-powerful computer processors" then you basically got smoked by Intel and other industry labs. But there's a different way to look at it. Because industry went off and solved the now-"boring" problem of building ever-more-powerful computer processors, th…

The "cool problem" is cracking the mystery of intelligence, of consciousness. Building a general AI. Until recently, this was mostly an academic pursuit, and people expected it to take decades or more. But now, suddenly, it's highly likely the problem will be cracked within a decade, and it will be done by corporate R&D teams , by means of scaling up the transformer architectures. So I get how they feel - one of the…

>> But now, suddenly, it's highly likely the problem will be cracked within a decade, and it will be done by corporate R&D teams, by means of scaling up the transformer architectures.

I'll counter that. In the end we need AI that can do training AND inference on edge devices out in the real world. A good (and possibly profitable) example would be robotic pets that can learn (even to understand words) and interact with their owners like real animals, but don't need to go to the vet or eat and poop. Big companies relying on huge compute resources are not even aiming at this type of thing. They're too busy using their "scale" to even bother looking at smaller but interesting methods or solutions.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#109

Post by gitfan86 »

johnaspden wrote: post #100
TeMPOraL wrote: post #95

Earlier quoted context omitted.

Eliezer and his disciples are looking at you and your friends, and he is not impressed.

Am Eliezer disciple, can confirm.

This day was inevitable once humans first told each other about their tools

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#110

Post by z3c0 »

foooorsyth wrote: post #106
bo1024 wrote: post #52

As a theorist in AI, the premise is funny but also really sad. And yet it is a very pervasive premise in the field. OpenAI is not doing science. They are building a big shiny thing, showing it off, keeping it closed, and making money off of it. That is not part of the scientific process. It is as if, every time SpaceX launched a rocket a little bit higher, every aerospace department ooh'd and aah'd and threw up their…

>That is not part of the scientific process What? OpenAI is absolutely conducting science amongst themselves. They are certainly engaging in the hypothesis -> test -> result scientific method, which is science in its purest form. Sharing the secret sauce is what is done or expected in academia , but that doesn’t mean that OpenAI isn’t conducting science behind their own walls. You’re conflating institutional academia…

"Science" is not solely the scientific method - it's the democratization of your methodology to enable the replication of the results, towards the goal of determining their soundness.
Post reply on HN