T0* – Series of encoder-decoder models trained on a large set of different tasks • Hacker News Board

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#161

Post by germandiago » Tue, Oct 19, 2021, 5:41 PM UTC

flipbrad wrote: post #65

germandiago wrote: post #44

Earlier quoted context omitted.

why? we should forbid arbitrary stuff based on political ideas or opinions?

I think that depends on the use of the technology in question. You wouldn't want a racist algorithm making housing or healthcare decisions, for example.

I am not who to qualify an algorithm as racist or sexist or anything. If I find it negative, I will not use it. I am not going to tell others what is what or what they can use or not.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#162

Post by danielcampos93 » Tue, Oct 19, 2021, 8:47 PM UTC

riku_iki wrote: post #142

philmcp wrote: post #29

This is fantastic progress, great to see 16x smaller = 41.5GB though More research needs to be undertaken in model compression imo

On superglue benchmark, much smaller Deberta outperforms vanilla T5: https://super.gluebenchmark.com/leaderboard I am curious why authors preferred T5?..

T5 has a notion of prompting. None of the *BERT have notions of prompting.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#163

Post by Aivean » Wed, Oct 27, 2021, 6:04 PM UTC

Mizza wrote: post #12

The hosted demo has the default query, "How many hydrogen atoms are in a water molecule?" It said "two". I asked it, "How many oxygen atoms are in a water molecule?". It said "two".

Hosted demo, "Logic puzzle" example: "On a shelf, there are five books: a gray book, a red book, a purple book, a blue book, and a black book. The red book is to the right of the gray book. The black book is to the left of the blue book. The blue book is to the left of the gray book. The purple book is the second from the right.

Which book is the leftmost book?"

Answer: > The black book

Same puzzle with the question "Which book is the rightmost book?"

Answer: > The black book

I tried to ask GPT-3 and Codex this problem, they could not solve it either.