Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Choose Your Weapon: Survival Strategies for Depressed AI Academics

arxiv.org

Post reply on HN
Print view View original

151–160 of 195 posts

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#151

Post by bheadmaster »

kome wrote: post #142
bheadmaster wrote: post #127

Earlier quoted context omitted.

I don't buy the argument that "public" science is the only real science. You can define it that way, but I don't see any reason to. "Public" science is just a special case of "restricted" science, such that the set of people the knowledge is restricted to is equal to the set of all humans.

I think this might get philosophical quickly, but to really understand where you're at, you need to access all knowledge. Everybody is standing on the shoulder of giants. It's a collective effort that happens through socialization and long-standing institutions like universities. An essential part of doing science is sharing results and waiting for others to challenge, expand, or falsify them. If you skip this step,…

I love to get philosophical, so I'm gonna throw this thought experiment here:

Let's say that there was a highly advanced civilization that existed way before the current civilized society. They performed scientific experiments and gained a lot of insight, but a catastrophical event erased them from the face of Earth together with all their knowledge.

Did that civilization engage in science?

If you think that they did, indeed, engage in science, then what is the difference between them and any other closed community that performs scientific experiments?

Or, if you think that they didn't, in fact, engage in science, then does that mean that science is only science if you and I, today, can gain access to the knowledge derived from the experiments? Therefore, science is defined by its utility to us?

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#152

Post by revelio »

bo1024 wrote: post #144
foooorsyth wrote: post #106

Earlier quoted context omitted.

>That is not part of the scientific process What? OpenAI is absolutely conducting science amongst themselves. They are certainly engaging in the hypothesis -> test -> result scientific method, which is science in its purest form. Sharing the secret sauce is what is done or expected in academia , but that doesn’t mean that OpenAI isn’t conducting science behind their own walls. You’re conflating institutional academia…

By the "scientific process", I mean the way scientific knowledge as a whole progresses over time, i.e. the main undertaking of the target audience of the paper. You're right that "OpenAI is not doing science" was hyperbolic, but I tried to clarify the aspect of science that I'm referring to.

Although this is a reasonable definition, it would mean many people who are referred to as scientists in practice would have to be considered non-scientists. Refusing to publish data and code is widespread in many academic fields, even in cases where academics sign documents promising to provide these things on demand.

For example, at minimum psychology, epidemiology and climatology have this problem. In climatology attempts to get the source code of models and data underlying papers has led to some of the biggest dramas in the field e.g. the Climategate hacker appears to have been motivated by the closedness of the field (similar to the llama leaker).

Given that society is clearly not ready to reclassify large parts of academic work as non-scientific, it seems like this definition cannot work even on its own terms.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#153

Post by mellosouls »

loandbehold wrote: post #133
mellosouls wrote: post #68

Earlier quoted context omitted.

it's quite plain it's an emergent property That is not at all quite plain. Respectable philosophical views include the proposition it is at least as fundamental as space or time.

Respectable to whom? That sounds like magic thinking, not science.

Philosophers. That's why I used the word "philosophical" not "scientific".

Dismissing serious ideas with thousands of years of philosophical history behind them as "magic thinking" is either a failure of imagination or education; you can rectify the latter by reading up on the subject.

https://en.wikipedia.org/wiki/Panpsychism

https://plato.stanford.edu/entries/panpsychism/

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#154

Post by revelio »

pronto-columnar wrote: post #147
VoodooJuJu wrote: post #132

Earlier quoted context omitted.

Nothing sad about it. Keep theorizing and academizing. It's useful. It's not nearly as useful as tinkerers building usable non-theoretical things. It's no wonder that launching a rocket is more impressive and eye-catching than a guy shuffling papers. Don't dare knock down the doers, the risk-takers, the ones with skin in the game. OpenAI provides me with a real, useful tool, something that directly improves mine and…

Weirdly aggressive response. do you think technologies like gpt were developed without any academic input?

Well, the core transformers/attention paper came from Google and the DNN revolution was also kicked off by Jeff Dean at Google, no? Yes, some of the very original neural net theory was laid in academia and maybe the original GAN paper came from it, don't recall, but AI academics were complaining about being unable to keep up and having their departments be raided even 7 or 8 years ago. It's not a new complaint. So this particular AI summer doesn't seem to owe much to academia.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#155

Post by wing-_-nuts »

melbourne_mat wrote: post #21

I can imagine AI academics are in a tough spot. However what about the existential angst the rest of us - who don't even do anything AI related on a day to day basis - are feeling? I think big changes are coming and it's not gonna be pretty.

My big question is what is your standard java / c# dev supposed to do to keep up with the massive changes that are coming? I was seriously considering doing a GT OMCS, but I don't know what sort of return on investment that would have for me.

I was also considering building a gaming pc to also be able to play around with ai / ml, only, I'm not even sure what one would be able to run locally anymore even with a 4090?

Overall? I feel you. It makes me worried for the new grads today. I don't really know if this old dog is up for new tricks.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#156

Post by ndriscoll »

staunton wrote: post #138
ndriscoll wrote: post #107

Earlier quoted context omitted.

You might be able to understand what a convolutional network "is" without calculus, but you'll be woefully unequipped to ask even obvious questions like "what if we put Fourier transforms around the convolutional layers" (a cursory search suggests it provides the expected speedup but is for some reason not a standard thing to do?). As someone outside of the industry, I'd also imagine any effort to explain what NNs ar…

> a cursory search suggests it provides the expected speedup What do you mean? Processing a CNN layer takes an amount of time that does not depend on the input data, only the input/output sizes. Fourier transform is just a change of basis. Why should anything speed up?

Because convolution is an O(N^2) operation, but a Fourier transform (and its inverse) can be done in O(NlogN) and turns convolution into O(N) multiplication. So if you do FFT, multiply, Inverse FFT, you get convolution in O(NlogN). I would guess that you don't even need to do the inverse FFT and can just learn in frequency space instead, but maybe there's some reason why that doesn't work out.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#157

Post by aiisjustanif »

numbers_guy wrote: post #30
Tomis02 wrote: post #20

Earlier quoted context omitted.

> Disregarding the fact that ~85% of machine learning academics do not have what it takes to be hired by OpenAI anyway That's interesting, could you please elaborate?

It's not interesting. You can regard OpenAI as one of the most prestigious AI labs in the world and only the best of the best get a chance to work there.

It’s very interesting that you think most people with a PhD working in AI / ML / Data Science academically can’t get a job at OpenAI.

You can easily see a lot of people who work at OpenAI on LinkedIn. University AI labs are always left out because non-commercial products just aren’t as noticeable.

I highly doubt that many of these academics would struggle being amount candidates at DeepMind, OpenAI and FAIR.

Prestige is very subjective, which makes it also very broad thankfully.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#158

Post by jimbokun »

chasd00 wrote: post #87
MrScruff wrote: post #35

Earlier quoted context omitted.

I think the mystery of why we're not just philosophical zombies was never going to have some eureka moment, given that it's quite plain it's an emergent property. Was there a creature in history that wasn't sentient that gave birth to a creature that was? Clearly not. The reason recent results are significant in this discussion is because: - Larger networks are already displaying emergent behaviours that couldn't hav…

Are you saying that consciousness didn’t result from, survival of the fittest, evolutionary pressure but more of a byproduct of a larger brain (network)? Like it emerged because of a larger network but was not selected for through evolution. That makes sense to me. Humans have a grotesquely large brain compared to other animals. Edit: I wonder if there are studies that have determined whether or not new born infants…

The large brain was selected for by evolution, which then gave rise to consciousness.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#159

Post by revelio »

anothernewdude wrote: post #28
visarga wrote: post #11

Earlier quoted context omitted.

I thought that too, a year ago. But then chatGPT Turbo came out (ten times cheaper), and a slew of 30B and 10B models that are decent. Now I believe we will be able to run non-trivial AI on trivial hardware. Not to mention the Stable Diffusion revolution, hardware requirements went down pretty fast. Even an old GPU from 5 years ago can generate images quickly. Some LLMs run on iPhones. The trick is always to offset t…

And? ChatGPT doesn't give you logprobs. It doesn't allow you to alter the generation algorithm. They've left their lunch out on the table for someone else to eat.

You can specify logit biases via the API, but yes it's limited.

Re: Choose Your Weapon: Survival Strategies for Depressed AI Academics

#160

Post by hyperbovine »

hyperbovine wrote: post #128
dmd wrote: post #104

Earlier quoted context omitted.

Yep. But we get paid at academic rates. I stay for the relaxed lifestyle, not the salary.

Then this post is not aimed at you, obviously.

Sorry, replied to wrong thread. :|
Post reply on HN