Use of Natural Language Understanding term in context of this benchmark is preposterous. No understanding takes place there. Please stick to NLP (Natural Language Processing) term for the next couple of decades. Thank you.
[deleted]
Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
131–140 of 246 posts
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#132I didn't know anything about SuperGLUE before (turns out it's a benchmark for language understanding tasks), so I clicked around their site where they show different examples of the tasks. One "word in context" task is to look at 2 different sentences that have a common word and decide if that word means the same thing in both sentences or different things (more details here: https://pilehvar.github.io/wic/ ) One of…
She landed a job. Got access to, acquired, etc.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#133Earlier quoted context omitted.
The second one means "the enemy successfully got several of our aircrafts". Specifically, definition 3a or 3b for the verb form here: https://www.merriam-webster.com/dictionary/land So potentially the enemy captured the aircraft (3a) or destroyed them (3b).
Would a native English speaker use the word "landed" in this way? In the context of aircraft? "Landed" is badly ambiguous here and several distinct meanings are plausible. Captured is the most natural word given your interpretation. Honestly that sentence -- the use of landed and that awful plural -- approaches engrish. Is that deliberate or is the use of English here just badly flawed? I can't see any other possibil…
There are a lot of native English speakers in the world and not all of them use the same idioms that you do. This seems like perfectly valid English to me; some other words that could be used instead of “landed” in the aircraft sentence include “bagged”, “nabbed”, “poached”, “got” and “did in”. One of the entertaining aspects of English is the multitude of ways it can be used.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#134I didn't know anything about SuperGLUE before (turns out it's a benchmark for language understanding tasks), so I clicked around their site where they show different examples of the tasks. One "word in context" task is to look at 2 different sentences that have a common word and decide if that word means the same thing in both sentences or different things (more details here: https://pilehvar.github.io/wic/ ) One of…
I think it's landed in the same sense as "landed a deal": got, or achieved, in this case achieving shooting them down. For me, my first read of the sentence would definitely be that it means shot down.
> I think it's landed in the same sense as "landed a deal": got, or achieved, in this case achieving shooting them down.
My buddy is a pilot and they always say "I landed the takeoff pretty good. PRETTY GOOD!"
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#135Earlier quoted context omitted.
Would a native English speaker use the word "landed" in this way? In the context of aircraft? "Landed" is badly ambiguous here and several distinct meanings are plausible. Captured is the most natural word given your interpretation. Honestly that sentence -- the use of landed and that awful plural -- approaches engrish. Is that deliberate or is the use of English here just badly flawed? I can't see any other possibil…
I think it's archaic; in the past a fishing reference would have been more common and widely understood.
I guess it annoys me because I suspect that if this is the sort of borderline incoherence one must wade through I would probably score below average.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#136Earlier quoted context omitted.
The second one means "the enemy successfully got several of our aircrafts". Specifically, definition 3a or 3b for the verb form here: https://www.merriam-webster.com/dictionary/land So potentially the enemy captured the aircraft (3a) or destroyed them (3b).
Would a native English speaker use the word "landed" in this way? In the context of aircraft? "Landed" is badly ambiguous here and several distinct meanings are plausible. Captured is the most natural word given your interpretation. Honestly that sentence -- the use of landed and that awful plural -- approaches engrish. Is that deliberate or is the use of English here just badly flawed? I can't see any other possibil…
I wouldn't use it that way myself, but at the same time the intended meaning is clear as day to me from the context. I'm surprised by the reactions. "Enemy" should give it away immediately.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#137Earlier quoted context omitted.
Ahh, just found an example where that's taken from https://glosbe.com/en/en/land . If you find on that page you'll see the exact sentence "the enemy landed several of our aircraft" (without the s after aircraft) which it says means "shoot down". I have still never heard landed used in that way, and again in other dictionaries I searched I couldn't find that definition either. Thus, this is a case where the "AI" may g…
I agree, AI should realistically be able to detect the rare/uncommon/ambiguous usage as well, and rated for that. I suppose in some case it could score better than humans on SuperGLUE benchmark.. but eventually it will have to come back down to near human score as it gets more accurate.
Why? In many of those benchmarks the average human score is not 100, but the AI progression doesn't really have a ceiling or a slow down at the human number. It should go through it and settle somewhere above. Plus we create these tests with our own limitations. There may be a world of more complexity or subtlelty that we all fail to grasp but the AI will.
I think humans are already behind at the face recognition task for example.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#138I didn't know anything about SuperGLUE before (turns out it's a benchmark for language understanding tasks), so I clicked around their site where they show different examples of the tasks. One "word in context" task is to look at 2 different sentences that have a common word and decide if that word means the same thing in both sentences or different things (more details here: https://pilehvar.github.io/wic/ ) One of…
Perhaps the enemy obtained several of our aircraft. In the same sense as one might land a new car in a contest.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#139I didn't know anything about SuperGLUE before (turns out it's a benchmark for language understanding tasks), so I clicked around their site where they show different examples of the tasks. One "word in context" task is to look at 2 different sentences that have a common word and decide if that word means the same thing in both sentences or different things (more details here: https://pilehvar.github.io/wic/ ) One of…
The second one means "the enemy successfully got several of our aircrafts". Specifically, definition 3a or 3b for the verb form here: https://www.merriam-webster.com/dictionary/land So potentially the enemy captured the aircraft (3a) or destroyed them (3b).
Aircraft typically get captured on the ground, or get forced to land by threat of being shot down. “Landed”, for me, would require the enemy to actively land the plane, just as “landing a fish” requires both the fisherman’s action and moving the fish from water to land.
I also wouldn’t use “landed” for destroying an enemy plane (neither by shooting it down nor by destroying it on the ground)
That, realistically, leaves hacking the plane’s electronics and then directing it to one’s own airfield.
Re: Google T5 scores 88.9 on SuperGLUE Benchmark, approaching human baseline
#140I didn't know anything about SuperGLUE before (turns out it's a benchmark for language understanding tasks), so I clicked around their site where they show different examples of the tasks. One "word in context" task is to look at 2 different sentences that have a common word and decide if that word means the same thing in both sentences or different things (more details here: https://pilehvar.github.io/wic/ ) One of…
Perhaps the enemy obtained several of our aircraft. In the same sense as one might land a new car in a contest.
Possible, but also the worst context to use land in. You land a car, but if the game host would say you landed a small airplane, there’d be a laugh from the crowd.