Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Choose Your Weapon: Survival Strategies for Depressed AI Academics

arxiv.org

Post reply on HN
Print view View original

171–180 of 195 posts

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#171

Post by pixl97 »

muskmusk wrote: post #82

> If we have learned one thing from deep learning, it is that scaling works. From the ImageNet [15] competitions and their various winners to ChatGPT, Gato [13], and most recently to GPT-4 [1], we have seen that more data and more compute yield quantitatively and often even qualitatively better results. (By the time you are reading this, that list of very recent AI milestones might very well be outdated.). Of course…

> Science isn't a competition to be won so that you can get praise and attention.

Conversely business seems to be a competition which wants exactly that, no matter what the long term consequences are.

Lead in gasoline makes the engine stop knocking, great. Problem solved, nothing to worry about, right?Don't worry about funding scientists to look into it, it's all going to be ok.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#172

Post by mellosouls »

loandbehold wrote: post #170
mellosouls wrote: post #153

Earlier quoted context omitted.

Philosophers. That's why I used the word "philosophical" not "scientific". Dismissing serious ideas with thousands of years of philosophical history behind them as "magic thinking" is either a failure of imagination or education; you can rectify the latter by reading up on the subject. https://en.wikipedia.org/wiki/Panpsychism https://plato.stanford.edu/entries/panpsychism/

There are many things respectable scientists believed thousand years ago that we now know are not true. Vitalist views of life and consciousness are one of those things. As of 2023, believing that consciousness is a distinct phenomenon as fundamental as space or time is a crackpottery at the level of believing 5G causes COVID.

That's a silly thing to say in the context of the links that will have instructed you differently.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#173

Post by staunton »

ndriscoll wrote: post #156
staunton wrote: post #138

Earlier quoted context omitted.

> a cursory search suggests it provides the expected speedup What do you mean? Processing a CNN layer takes an amount of time that does not depend on the input data, only the input/output sizes. Fourier transform is just a change of basis. Why should anything speed up?

Because convolution is an O(N^2) operation, but a Fourier transform (and its inverse) can be done in O(NlogN) and turns convolution into O(N) multiplication. So if you do FFT, multiply, Inverse FFT, you get convolution in O(NlogN). I would guess that you don't even need to do the inverse FFT and can just learn in frequency space instead, but maybe there's some reason why that doesn't work out.

I see, the wording confused me. You don't really "put Fourier transforms around the convolutional layers" because you have to completely replace the convolutions.

This seems to be done in some cases. I guess it isn't done more widely because the "standard" convolution kernels are very small and the performance would actually be worse?

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#174

Post by jondea »

ndriscoll wrote: post #156
staunton wrote: post #138

Earlier quoted context omitted.

> a cursory search suggests it provides the expected speedup What do you mean? Processing a CNN layer takes an amount of time that does not depend on the input data, only the input/output sizes. Fourier transform is just a change of basis. Why should anything speed up?

Because convolution is an O(N^2) operation, but a Fourier transform (and its inverse) can be done in O(NlogN) and turns convolution into O(N) multiplication. So if you do FFT, multiply, Inverse FFT, you get convolution in O(NlogN). I would guess that you don't even need to do the inverse FFT and can just learn in frequency space instead, but maybe there's some reason why that doesn't work out.

Computing convolutions using FFTs is efficient for large kernels (or filters). Most convolutions in popular ML models have small kernels, a regime where it is typically more efficient to reformulate the convolution as a matrix multiplication.

I think your complexity argument is correct for N=pixels=kernel size. But typically, pixels>>kernel size.

Disclosure: I work at Arm optimising open source ML frameworks. Opinions are my own.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#175

Post by ericmcer »

melbourne_mat wrote: post #21

I can imagine AI academics are in a tough spot. However what about the existential angst the rest of us - who don't even do anything AI related on a day to day basis - are feeling? I think big changes are coming and it's not gonna be pretty.

Can't they just jump into one of the thousands of new high paying jobs that have been created for their specific niche in the last few months? I am having trouble being sympathetic to AI researchers that AI is exploding.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#176

Post by Rufbdbskrufb473 »

elif wrote: post #59
southernplaces7 wrote: post #24

Earlier quoted context omitted.

Nothing about current AI models profoundly challenges the unknowns of what makes consciousness work in any mystery-resolving way. People who confuse plainly programmed algorithmic systems like GPT 4 or Midjourney with the still unresolved issues of sentience as we know it so far are drinking far too much of the current batch of AI punch. Sadly, it's a sentiment I see all too often on HN, a site in which i'd assume mo…

Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.

Why?

Isn't most ethical behavior driven by self-interest? We want to follow general principles that promote respecting those who are similar to us in the hope that others follow the same principles when interacting with us.

AI is too dissimilar for us to worry about whether its "mistreatment" would make it more likely for others to start mistreating ourselves.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#179

Post by elif »

nradov wrote: post #137
elif wrote: post #59

Earlier quoted context omitted.

Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.

Why should I respect silicon? It's just a tool, like a screwdriver.

you're arguing with Arthur C Clarke.
Post reply on HN