An "AI Breakthrough" on Systematic Generalization in Language?
aiguide.substack.com
An "AI Breakthrough" on Systematic Generalization in Language?
1–6 of 6 posts
Re: An "AI Breakthrough" on Systematic Generalization in Language?
#21) https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
2) Study used 25 people
3) via Mechanical Turk
Interesting topic but I don't see a breakthrough.
Re: An "AI Breakthrough" on Systematic Generalization in Language?
#3[deleted]
Re: An "AI Breakthrough" on Systematic Generalization in Language?
#41) https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline... 2) Study used 25 people 3) via Mechanical Turk Interesting topic but I don't see a breakthrough.
Why would it matter? They wanted data generated for comparison and got it. They wanted a common type of human error and got it too. What's the criticism exactly?
Re: An "AI Breakthrough" on Systematic Generalization in Language?
#5Very interesting write up and analysis of a significant paper. The blog's author has herself made fascinating contributions in AI (especially on abstraction). I find it very helpful to read these kinds of in depth reviews.
Re: An "AI Breakthrough" on Systematic Generalization in Language?
#61) https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline... 2) Study used 25 people 3) via Mechanical Turk Interesting topic but I don't see a breakthrough.
Why would it matter? They wanted data generated for comparison and got it. They wanted a common type of human error and got it too. What's the criticism exactly?
Yes, they got some data, and got some errors. But they are using 25 samples to characterize "human reasoning". Sure, it's a place to start.
The bigger issue is that this pattern - small samples, via Mechanical Turk - is a frequent flier in papers that make claims that end up failing further scrutiny. It's more common in sociology and psychology than AI research, but I think we know this isn't a solid foundation to build a lot of extrapolation on.