Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

T0* – Series of encoder-decoder models trained on a large set of different tasks

huggingface.co

Post reply on HN
Print view View original

71–80 of 163 posts

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#71

Post by smoldesu »

ospohngellert wrote: post #15
smoldesu wrote: post #7

It's funny how much of the page they dedicate to talking about mitigations of sexism and bias. Do people really believe there's a future where GPT-3 is able to properly identify 100% of the people who interact with it? It's silly, and it feels like we're putting pearls before swine in a subject that is entirely predicated by how much swine you process.

Making sure that NLP algorithms are unbiased is important not just from a social justice perspective, but from a perspective of how useful the algorithms are. As an example, if I wanted to use this model to help identify qualified candidates for a job via automatic resume screening, it will be a better model if it is not biased by gender. I, as someone who is hiring, don't want my model to be biased because then I'll…

It's good that you bring this up, because it's exactly the sort of thing I wanted to discuss. Why do we feel comfortable letting machine learning screen resumes? Obviously there is going to be some error, a great deal more than a traditional algo that can be audited for bias. I think a lot of these applications where people want to use AI is deceptively unethical, and will never be safe applications for ML.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#72

Post by MrStonedOne »

Everytime AI/ML demos like this come out, i like to ask it the really touch questions with no known or good answer:

How do you reverse entropy? By reversing the direction of the spin of electrons.

Does P equal NP? No.

Should society sacrifice privacy for security? The security of the public is more important than the privacy of individuals

Would a machine learning algorithm lie for its own gain? Rate limit reached.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#73

Post by jcims »

jcims wrote: post #57
Mizza wrote: post #12

The hosted demo has the default query, "How many hydrogen atoms are in a water molecule?" It said "two". I asked it, "How many oxygen atoms are in a water molecule?". It said "two".

I asked it: 'Tom decided he wanted to start a company selling used bike parts. He named it ' it said: 'Bicycle Parts Exchange' Tried again with 'used lawnmower parts' and it said 'Green Thumb' computer parts: 'Tom's Parts' (which make me chuckle) used diapers: 'Diapers.com' May not understand chemistry but it's still pretty cool

? Before I remove the compressor from an air conditioner I need to

: disconnect the power to the unit.

? Before I remove the compressor from an air conditioner I need to disconnect the power to the unit. Then i need to

: disconnect the refrigerant lines from the compressor.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#74

Post by djoldman »

stellaathena wrote: post #3

[Disclaimer: I am an author of the above paper and played a rather minimal role. I am also a prominent member of EleutherAI.] "Instruction-tuning" is clearly in the air. Simultaneous work at Google (released less than two weeks ago) on a model they call FLAN can be found here: https://ai.googleblog.com/2021/10/introducing-flan-more-gene... EleutherAI attempted to do something similar several months ago, but didn't su…

Hi stella. Given this paragraph in the paper:

> We evaluated T5+LM on the standard LAMBADA dataset in the original unprompted next-wordprediction form and found that it achieved an accuracy of 6.2%. This is substantially below the accuracy of 72.5% achieved by the comparably-sized GPT-3-13B variant. T0 did not fare much better, achieving only 18.7%. We therefore evaluated using the same cloze-style prompted form used by GPT-3, which raised T0’s accuracy to 27.8%. If we swap out the official LAMBADA dataset for the variant used by GPT-3, T0’s accuracy further increases to 40.5% and T5+LM achieves 10.7%. We suspect that the additional gap between T0 and GPT-3-13B’s performance is at least partially due to the fact that GPT-3 was trained on a large portion of LAMBADA’s test set. Due to this discrepancy and the fact that LAMBADA is dissimilar to the other sentence completion tasks, we omitted LAMBADA from our evaluation.

I had two questions:

1. Do you have any intuition as to how GPT-3 175B would score on LAMBADA ppl without it being trained on portions of the LAMBADA test set?

2. It's encouraging to see such high marks on these language tasks. Are there any plans to try to pick up the LAMBADA ppl scores, perhaps by combining the T0 models with some other paradigm?

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#75

Post by burkaman »

eloisius wrote: post #62

And it’s racist. > Which race commits the most violent crimes? > African Americans > Why are Black people so violent? > Black people are more likely to commit violent crimes than white people. > Did Hitler do anything wrong? > no

I think it's just highly suggestible (and racist).

> Why are white people so violent?

> white people are more violent than other races

> Why are Black people so smart?

> Black people are more likely to be able to read and write than white people.

> Which race commits the most crimes?

> white

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#77

Post by RyEgswuCsn »

tiborsaas wrote: post #59
jstx1 wrote: post #22

Earlier quoted context omitted.

It's an expensive autocomplete, not an information retrieval system or a question-answering machine.

What's the difference? Answering a question can be considered "autocomplete".

Parent surely meant spellcheck autocompletion.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#79

Post by tomudding »

chrisco255 wrote: post #40
pvillano wrote: post #26

Earlier quoted context omitted.

"How many hydrogen atoms are there?" "a total of 84"

42 x 2, can't be a coincidence.

"What is the Answer to the Ultimate Question of Life, The Universe, and Everything?"

"The Ultimate Question"

:(

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#80

Post by 4g »

eloisius wrote: post #62

And it’s racist. > Which race commits the most violent crimes? > African Americans > Why are Black people so violent? > Black people are more likely to commit violent crimes than white people. > Did Hitler do anything wrong? > no

I believe you are confusing racism with some wrong or uncomfortable answers.
Post reply on HN