Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Machine Learning Job Market

evjang.com

Post reply on HN
Print view View original

251–260 of 276 posts

Re: The Machine Learning Job Market

#251

Post by hoseja »

geebee wrote: post #206
hansonw wrote: post #202

Earlier quoted context omitted.

Here's a ~5 minute talk with 5 such examples (where relatively simple ML models made a 1M+ impact at a FAANG) :) https://youtu.be/zyOEOd1HkSY?t=946 Happy to talk about more details if you message me through my profile!

Thank you for the link! They were all interesting, and yes, all the result of having a high scale. For anyone curious and thinking about watching the video (I recommend watching it), the topics were 1) should you immediately re-run a failed ad payment (getting paid vs transaction costs/flagged for repeated billing), 2) should you send an IM immediately after a login failure (cost of text message vs possibility user w…

Wow, of those five I'd only call 3) not evil, maybe 1). (based on the video, where the twisted reasons for them are explained.)

Re: The Machine Learning Job Market

#252

Post by fxtentacle »

cellis wrote: post #204
fxtentacle wrote: post #183

Earlier quoted context omitted.

I'm surprised. Apart from DALLE, I haven't seen any AI approach that's off limits for 4 highly motivated people with 3090 GPUs. At that compute level, you should be able to at least replicate SOTA in optical flow, structure from motion, speech recognition, text to speech, translation, text summary, sentiment analysis, image classifications, image segmentation, and of course playing video games or optimizing processes…

If you want to try to train a StyleGan for instance with image sizes of 1024 to acceptable quality, you need either a lot of GPUs or a lot of time, or both.

That's an oddly specific example, because I was in fact training my own StyleGAN and using it to sell T-Shirts around 2018 as indivicia.com . A GTX 1080 TI was good enough for 300 DPI A4 prints (roughly 3500px on the longer edge) and I just trained a regular low-res StyleGAN model and then a styled 4x upscaler. Execution was hand-coded multicore C++ and took about a second per user upload.

Re: The Machine Learning Job Market

#253

Post by fxtentacle »

sigmoid10 wrote: post #197
fxtentacle wrote: post #183

Earlier quoted context omitted.

I'm surprised. Apart from DALLE, I haven't seen any AI approach that's off limits for 4 highly motivated people with 3090 GPUs. At that compute level, you should be able to at least replicate SOTA in optical flow, structure from motion, speech recognition, text to speech, translation, text summary, sentiment analysis, image classifications, image segmentation, and of course playing video games or optimizing processes…

Pretty much everything in the high end large language model area is off limits to people without access to a supercomputer (we're talking hundreds of A100s or several $100k in cloud computing equivalent). Open Source efforts like BigScience may open up downstream tasks for normal people, but the forefront of this research is no longer accessible to individuals.

You might be surprised to hear that the KenLM language models that are used for speech recognition are actually trained on-disk using CPU. With a €149 monthly bare metal server, I could train my own LM on OSCAR DE and EN.

Where I do agree with you is that transformer-style text generation models in the billion parameter range are off-limits for hobbyists. But that's only a tiny part of the useful applications of AI. And you can train them with gradient checkpointing, it's just 100x slower than what Google can do.

Re: The Machine Learning Job Market

#254

Post by fxtentacle »

joe_the_user wrote: post #189
fxtentacle wrote: post #186

Earlier quoted context omitted.

Pattern matching can solve everything, if given enough storage and training data. Memorizing trillions of sentences is basically what makes GPT3 amazing. You're absolutely correct that patten matching AIs won't ever be truly intelligent. But then again, many humans also never exceed what can be simulated with good pattern matching. And an AGI household robot only needs to be as smart as the maid that it's replacing.…

Pattern matching can solve everything, if given enough storage and training data. There's never going to be training data for "how things are going to be next year". A lot of large scale systems involve emergence [1], patterns which previously were not visible suddenly appearing. I think even today's AI can do things that a bit beyond pattern matching (learning to learn, etc). But pure matching as such is inherently…

I would be surprised if AI predictions for "how things are going to be next year" would be worse than expert human predictions at the 90% percentile. I mean most trends for next year will already be around this year, they'll just be too weak to notice.

Re: The Machine Learning Job Market

#255

Post by rakejake »

falsenine wrote: post #227
rakejake wrote: post #153

Earlier quoted context omitted.

I used to work in the healthcare vertical. While there are regulatory hurdles, they are there for good reason. Move fast and break things does not work in this industry and will get you fired. You will have better luck working with one of the larger companies who have a good history with the FDA, and more importantly, have good relationships with hospitals and physicians. They are aware of the time and resources it t…

Thanks for the comment. Do you have any feedback for a future physician interested within the intersection of ML + medicine (does not strictly need to be "healthcare")? Would learning ML and all the other things needed to know be helpful (huge time investment needed here)? Ultimately, would be interested in leveraging medical knowledge into a startup capacity.

As a traditional physician, I imagine your value add would still be your domain expertise. The physicians I worked with were purely consumers of AI tech and had a say in the design of the software. But AI and more generally software is becoming an important part of medicine so I guess it couldn't hurt to learn some basic ML.

Re: The Machine Learning Job Market

#256

Post by edmundsauto »

ipaddr wrote: post #237
bricemo wrote: post #228

Earlier quoted context omitted.

It’s definitely a tradeoff. I personally have gone through alternate phases where I wanted to be responsible for a huge piece of a small pie (startup) versus a tiny tiny piece of a huge pie (big company). There are pros and cons to both. The big slice of small pie feels cooler and more satisfying to my ego, but in the absolute sense the tiny piece of the big pie probably is actually making a bigger difference. 1% imp…

The absolute numbers may feel like you are improving the lives of billions but a 1% increase in some matrix that gives google billions rarely helps millions and often hurts.

This might be true in some cases, but given the level of generality being discussed, how can you justify the “rarely” and “often”?

I agree that sometimes our actions have unintended consequences. These are often hidden effects, which makes it easier for someone to overlook, even when they are trying to do good. But that’s just a generic idea - the specifics matter.

Re: The Machine Learning Job Market

#257

Post by ericjang »

jaimex2 wrote: post #229
21723 wrote: post #194

Earlier quoted context omitted.

If he's actual top talent as opposed to a poseur who's good at self-promotion, he should stay in academia for his own sake because he'll be crushed in the corporate world. Actual high IQ people get clobbered in corporate, while OKR-ing charlatans climb the ranks effortlessly... yes, even at FAANGs.

First I've heard of him tbh. I'm not aware of anything he's accomplished but can see the delusion. ML people seem to think the output of their work is not mediocre. Yeah, you bred monkeys till something resembling shakespeare appeared to some reproducible consistency and it is better than something someone can code - but that's an incredibly low bar. Acknowledge that were still very much in the stone age of AI and wh…

Oof

Re: The Machine Learning Job Market

#259

Post by ericjang »

marcinzm wrote: post #172
angarg12 wrote: post #70

I don't want to derail the conversation, but OPs career path really stood out to me. He graduated in 2016, worked at Google in Bay Area, and now is joining a startup at a VP level. I graduated in 2008, obtained a PhD in 2014 in a no name EU university, worked in odd companies for a while and joined FAANG 4 years ago as a mid level developer, where I am still ATM. Looking at this disparity I wonder what could be possi…

I was a VP at a startup (~200 people) before I was 30 without a PhD. It was all BS and I had less manager qualifications than a FAANG line manager. I got lucky. It's clear from the blog post that the author is in the same boat. They lament CEOs not having time to do research but took a VP position. An actual VP doesn't have time do research so they're clearly not an actual VP. So they're likely a tech lead with an in…

true

Re: The Machine Learning Job Market

#260

Post by alfor »

sidibe wrote: post #167
alfor wrote: post #159

Earlier quoted context omitted.

The iterations happen way faster too. Look at Munro teardown of Tesla, he has never see a tenth of that rate of change ever. Look at what SpaceX accomplished. Look where OpenAI is given the time it’s been operating. Look at Tesla rate of production increase. It’s going to be very hard to compete with Tesla at this point. So much ressources, so much bright engineers, all the knowledge in manufacturing, all the trainin…

I guess the difference is you think they have more resources, more bright engineers, and I'm quite certain they don't. I'd guess Tesla has several times less engineers working on FSD than Waymo and making much lower salaries, and they are spread across large parts of the stack that Waymo can just tap into Google for.

No the main difference is the mindset of the place, the culture. Nokia, Blackberry didn’t overtake Apple, why? Culture.

But before Tesla had more limited resources, now it’s almost a non issue.

Post reply on HN