No, but it is weird to use "modern" here. Modern suggest a longer timeframe. I would say machine learning deep NNs is modern AI. It's just not true that everything not transformer is outdated, but it is "kinda" true that everything not deep NN is outdated.
> It's just not true that everything not transformer is outdated. Am not expert, do you have some links about this? i.e. a neural net construction that outperforms a transformer model of the same size.
ConvNets Match Vision Transformers at Scale (Oct 2023)
Why would that be of note, especially in America? I think it would be an interesting observation in China or Japan, or some other country which is generally less welcoming to immigrants than the US
First generation immigrants are still a tiny minority of the population. The fact that the entire team consists effectively of first generation immigrants says something, probably both about higher education and American culture.
Isn't some of it have to do with it being a self selecting sample? If you come to America to study, you were a good student in your resident country leading to more chances of success than the average local citizen. Their children might be smarter on average.
Alternatively if you are coming fleeing persecutions you are enterprising enough to set up something for your children. That hard work inculcates a sort of work ethic in such children which in turn sets them up for success.
Speaking from experience as an immigrant myself.
Why would that be of note, especially in America? I think it would be an interesting observation in China or Japan, or some other country which is generally less welcoming to immigrants than the US
First generation immigrants are still a tiny minority of the population. The fact that the entire team consists effectively of first generation immigrants says something, probably both about higher education and American culture.
Transformer == Modern AI? Wow, I dont know where should I start My mind literally stuck at the absurdity
Modern AI != Machine Learning. Artificial intelligence has always been about models being able to accomplish a broad swath of tasks that they weren't explicitly trained on. It seems hard to argue we had accomplished this before massive transformers came along.
And none of them still work for Google. It’s truly baffling that Google’s CEO still has a job after how badly he fumbled on AI.
Their wildest success at Google would have meant the company's stock price doubling or tripling and their stock grants being worth a couple million dollars at most. Meanwhile all of them had blank checks from top VCs in the valley and the freedom to do whatever they wanted with zero oversight. What exactly could Sundar have done to retain them?
> First generation immigrants are still a tiny minority of the population Not in California. Last I remember, something like a quarter of the state’s population is foreign born.
That's still a 1-in-4^8 = 1-in-65536 longshot.
Its really not; hiring within a single firm, especially for related functions, will tend fo have correlated biases, rather than being a random sample from even the pool of qualified applicants, much less the broader population.
Of note: “ Six of the eight authors were born outside the United States; the other two are children of two green-card-carrying Germans who were temporarily in California and a first-generation American whose family had fled persecution, respectively.”
Why would that be of note, especially in America? I think it would be an interesting observation in China or Japan, or some other country which is generally less welcoming to immigrants than the US
> less welcoming to immigrants than the US
Not sure if we can claim this any more, what with texas shipping busloads of immigrants to NY and the mayor declaring it a citywide emergency, and both major parties rushing to get a border wall built.
Why would that be of note, especially in America? I think it would be an interesting observation in China or Japan, or some other country which is generally less welcoming to immigrants than the US
Because there are plenty of people in the US who are neither immigrants nor the children of immigrants. In fact, they're probably a significant majority. So to have 8 out of 8 be members of the smaller set is rather unlikely.
Not when you consider that those people were pulled from a worldwide talent pool for a relatively niche topic. If you can recruit anyone you want in the world and end up with 8/8 Americans, that would be weird.
Attention models? Attention existed before those papers. What they did was show that it was enough to predict next word sequences in a certain context. I'm certain they didn't realize what they found. We used this frame work in 2018 and it gave us wildly unusual behavior (but really fun) and we tried to solve it (really looking for HF capability more than RL) but we didn't see what another group found: that scale in compute with simple algorithms were just better. To argue one group discovered and changed AI and ignore all the other groups is really annoying. I'm glad for these researchers. They deserve the accolade but they didn't invent modern AI. They advanced it. In an interesting way. But even now we want to return to a more deterministic approach. World models. Memory. Graph. Energy minimization. Generative is fun and it taught us something but I'm not sure we can just keep adding more and more chips to solve AGI/SGI through compute. Or maybe we can. But that paper is not written yet.