211–220 of 276 posts
Earlier quoted context omitted.
This describes 95% of machine learning at FAANG+, unfortunately nobody likes to talk about. Context: I work at a FAANG.
God, and it's such a snooze-fest. Writing TFX code at Google is like having your soul-sucked through your rear-end! Imagine TF1 with all the broken APIs, but now it's all distributed! Fun.
I laugh-snorted reading that! I am stealing that phrase.
"FAANG+similar : Low 7 figures compensation (staff level), technological lead on compute (~10 yr)" I don't know where OP is getting these figures from, but I doubt that FAANGs offer 7-figure comps to Staff-level people. It's probably more in the higher 6-figure level (400K - 600K).
The author is a skilled research scientist in a very competitive space with some high-profile publications (e.g. Gumbel Softmax). He is absolutely an outlier, but not a unicorn -- AI researchers with good publications and reputation will attract a lot of interest from companies with lots of money to spend. Low 7-figures for an ~L7 research scientist with competing offers from FAANG research labs is not crazy.
Earlier quoted context omitted.
These outliers are rare but they do exist. I knew someone at Google who was hired as L3 straight out of college (as all non-PhDs are) and got promoted once a year to L6 (Staff) so 3 years. He got promoted to L7 2 years after that. It's a rare combination of talent and the right circumstances but it does happen.
I tried to hint at this by using quotes, I don't doubt that L6 is possible. But, please elucidate, are there L6s at Google making "low 7 figures"? From levels.fyi, there are no such reports. The average is about half and matches what I know from other companies. Those that are approaching 7 figures have at least a decade of experience. Anyway, the pay he describes is much closer to L8.
Low 7 figures is realistic for L8s without the share price noticeably appreciating. With discretionary grants some L7s may squeak into that club. But L6? No way.
This post feels like author is insecure about his position and wants to establish some validity. Having going through it all, it feels delusional at best. The glorified pattern matching can only take us so far. You know it's working as long as there is a pattern. I wouldn't call it a general intelligence per se. There is no "juice" in these algorithms. If we use these tools, we can immediately see where they fail and…
Pattern matching can solve everything, if given enough storage and training data. Memorizing trillions of sentences is basically what makes GPT3 amazing. You're absolutely correct that patten matching AIs won't ever be truly intelligent. But then again, many humans also never exceed what can be simulated with good pattern matching. And an AGI household robot only needs to be as smart as the maid that it's replacing.…
What we need is a pattern detector + a the ability to create basically infinite ANNs (or be able to multitask on them) + an event loop that takes input feeds (from cameras, microphones etc,) does some kind of reasoning and then pushes to its output feeds (wheels, etc.)
I think you use pattern matching to extract unique objects, store these objects as a node with its own simple neural net + long term storage where it only stores pictures of this object plus a dataset about it e.g, how often you see it. You then you organize them into an object hierarchy. Each new object is compared against all other objects we’ve stored using their pattern marchers. The higher the output the more weight we give their “connection.” Each object is made up of of sub objects so they are the top of their own tree as well, so you can run this pattern finder on the dataset of individual objects itself and if you find new objects the tree recurses. You can then check these objects against existing ones etc.
A general intelligence does this constantly, in real time. Then it’s a quick algorithm.
1. Have I seen this object before
2. No, but it shares characteristics with animals (an object that groups together all things that look like animals.)
3. It’s much larger than my dog, and I’ve seen large animals attack small ones more often than not.
4. My dog is also a dick, and attacks other animals more often than not
5. It’s probably a threat
Just scaling modern compute won’t get you there unless you’re willing to dedicate a few orders of magnitude more energy than a human being to do so. You need a completely different, distributed, architecture if you are going to be able to compare billions of objects against billions of objects every time you see something new and in real time.
Machine Learning is great but it’s only the learning part. Intelligence is reasoning about multiple things in relation to one another not detecting a pattern. You might trick yourself into thinking you’re getting there because pattern matching is powerful but it’ll get you to the intelligence of microbe at best. Even then you need something that’s driving the actions.
"FAANG+similar : Low 7 figures compensation (staff level), technological lead on compute (~10 yr)" I don't know where OP is getting these figures from, but I doubt that FAANGs offer 7-figure comps to Staff-level people. It's probably more in the higher 6-figure level (400K - 600K).
looking at levels.fyi it seems there are at least several companies paying 7 figures to ML engineers with less than 10 years experience, although majority are at 15+ https://www.levels.fyi/Salaries/Software-Engineer/Machine-Le...
Earlier quoted context omitted.
Eric Jang is top ML talent, these numbers are accurate. I work in ML and have followed his work for years
If he's actual top talent as opposed to a poseur who's good at self-promotion, he should stay in academia for his own sake because he'll be crushed in the corporate world. Actual high IQ people get clobbered in corporate, while OKR-ing charlatans climb the ranks effortlessly... yes, even at FAANGs.
> The most important deciding factor for me was whether the company has some kind of technological edge years ahead of its competitors. A friend on Google’s logging team tells me he’s not interested in smaller companies because they are so technologically far behind Google’s planetary-scale infra that they haven’t even begun to fathom the problems that Google is solving now, much less finish solving the problems that…
> The most important deciding factor for me was whether the company has some kind of technological edge years ahead of its competitors. A friend on Google’s logging team tells me he’s not interested in smaller companies because they are so technologically far behind Google’s planetary-scale infra that they haven’t even begun to fathom the problems that Google is solving now, much less finish solving the problems that…
Let me ask you a question -- what proves your ability to work on scale as an engineer? Is it the scale of the problem, or the scale of your solution?
Outside of "Google scale" logging is a nearly trivial, solved problem. If the only thing that creates the "challenges of scale" is your data cardinality, guess what. You're only solving problems of scale in the most literal (and maybe trivial) sense. Working on that at Google isn't going to prepare you for how to architect a platform for a startup that scales from 0 to 1 million active users over night without breaking a sweat.
I am biased in that I care about the latter kind of scale and effectively couldn't care less about the former, because the latter is generally an existential problem to have, while the former is a nice problem to have.
Earlier quoted context omitted.
On the other hand, there’s a lot of real problems that real people actually deal with that just need a logistic regression to save million bucks here and there. I like that space more.
If you have a few minutes, can you list a few of these problems? Just curious here!
A common thing I do is say company X has a fleet of Y assets. They repair them every N years. A good solution would be to predict which ones need repairs. Do those ones more often. Do the healthy ones less often. Pay more attention to the ones that are valuable.
Better outcomes, millions less spending. Probably don’t even need a live model in prod. Just a semi annual manual export run to excel for some planner guy who’s been keeping the schedule for 2 decades