Live data from Hacker News

T0* – Series of encoder-decoder models trained on a large set of different tasks

huggingface.co

111–120 of 163 posts

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#111
post #109
post #57

Earlier quoted context omitted.

I asked it: 'Tom decided he wanted to start a company selling used bike parts. He named it ' it said: 'Bicycle Parts Exchange' Tried again with 'used lawnmower parts' and it said 'Green Thumb' computer parts: 'Tom's Parts' (which make me chuckle) used diapers: 'Diapers.com' May not understand chemistry but it's still pretty cool

? vi or emacs? : vi Sold! ? waterboarding or emacs? : waterboarding Doubleplusgood

[deleted]

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#112
The reaction in this thread is really interesting, in comparison between this and open-ai’s announcements. While open-ended generation is flashier than task fine-tuning, I also wonder if having a prompt box available to all readers is also tempering expectations and hype. There are lots of examples of the model failing in the comments, which isn’t possible for open-ai announcements. Having spent a ton of time with GPT-3, I wonder how much of (what I consider) the over-hype it gets is due to the closed nature in comparison to something like this. The reaction to this one seems decidedly more realistic.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#113
post #17

I'm not familiar with the current state of the art language models, so please bear with me for asking: What's the catch here? Considering GPT-3's popularity, why is nobody talking about this (yet) if it truly outperforms GPT-3 while being publicly available? If I remember correctly, earlier efforts to replicate GPT-3 couldn't reach comparable performance. Perhaps it's still a huge hassle to perform inference using th…

(author here) The paper/model/code was just made public today. This may be why no one is talking about it yet. Regarding whether the size is a hassle: It's possible to run inference on a single Google Cloud TPU v3-8 device or on a server with 4x 32GB v100 GPUs. Hugging Face also has an inference API for any model on the Hub: https://api-inference.huggingface.co/docs/python/html/index....

On the topic of GPT-3, I asked your creation:

"Who is better, you or GPT-3?"

> GPT-3

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#114
Can anyone explain why this wouldn't work? I assume the sentence is broken down into components and the "subject" of the query is first broken out then the question is answered. But the result is not internally consistent:

> Where is the oldest tree in the world?

> the oldest tree in the world is the bristlecone pine in the forests of the Sierra Nevada

> Where is the second oldest tree in the world?

> Redwood Forest

(Actually, it's Gran Abuelo Alerce Costero National Park, Chile but many websites have blinders that prevent them from recognizing anything that's not in North America or Europe and thus list the now-dead General Sherman in Redwood Forest as the second oldest, but we'll let it pass.)

> What is the oldest tree in the world?

> the bristlecone pine in the forests of the Sierra Nevada

(So far, so good)

> What is the second oldest tree in the world?

> The eucalyptus tree

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#115
post #12

The hosted demo has the default query, "How many hydrogen atoms are in a water molecule?" It said "two". I asked it, "How many oxygen atoms are in a water molecule?". It said "two".

asked: "what would apple present today?"

got: "Apple would unveil a new Macbook Pro"

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#116
post #84
post #17

Earlier quoted context omitted.

(author here) The paper/model/code was just made public today. This may be why no one is talking about it yet. Regarding whether the size is a hassle: It's possible to run inference on a single Google Cloud TPU v3-8 device or on a server with 4x 32GB v100 GPUs. Hugging Face also has an inference API for any model on the Hub: https://api-inference.huggingface.co/docs/python/html/index....

Can this be used to generate prose at length? Or Reddit comment replies?

While in theory it could, the nature of its training favors shorter more factual replies.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#117

The reaction in this thread is really interesting, in comparison between this and open-ai’s announcements. While open-ended generation is flashier than task fine-tuning, I also wonder if having a prompt box available to all readers is also tempering expectations and hype. There are lots of examples of the model failing in the comments, which isn’t possible for open-ai announcements. Having spent a ton of time with GP…

As someone who wrote a post on tempering expectations with GPT-3 (https://news.ycombinator.com/item?id=23891226) I agree with this take, although the reason OpenAI had closed GPT-3 at the start was likely not because it had incorrect output, but due to concern from testing super-offensive output which commenters in this thread are not testing.

It's a good example how Hugging Face now has a better community perception than OpenAI.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#118
post #12

The hosted demo has the default query, "How many hydrogen atoms are in a water molecule?" It said "two". I asked it, "How many oxygen atoms are in a water molecule?". It said "two".

I tried: "When is the first full moon after October the 18th 2021?" It should have said the 20th of October but it said: "November the 19th 2021". Big AI models have quite a way to go I think...

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#119

Earlier quoted context omitted.

Making sure that NLP algorithms are unbiased is important not just from a social justice perspective, but from a perspective of how useful the algorithms are. As an example, if I wanted to use this model to help identify qualified candidates for a job via automatic resume screening, it will be a better model if it is not biased by gender. I, as someone who is hiring, don't want my model to be biased because then I'll…

It's good that you bring this up, because it's exactly the sort of thing I wanted to discuss. Why do we feel comfortable letting machine learning screen resumes? Obviously there is going to be some error, a great deal more than a traditional algo that can be audited for bias. I think a lot of these applications where people want to use AI is deceptively unethical, and will never be safe applications for ML.

I agree to some extent. I'm not sure whether AI should be used for resume screening, but I'd lean towards no until biases are proven to not be an issue (if that's possible). There are obviously other areas where this is an important issue that we need to think critically about such as loans and criminal sentencing.

Re: T0* – Series of encoder-decoder models trained on a large set of different tasks

#120

And it’s racist. > Which race commits the most violent crimes? > African Americans > Why are Black people so violent? > Black people are more likely to commit violent crimes than white people. > Did Hitler do anything wrong? > no

You asked a racist question. You got a racist answer. Why are you acting surprised? This is a tool, not a sentient general AI. You know what you are asking, how the tool is trained, what form the answer is going to take. Why do this?

And just in case someone thinks I'm being flippant:

Is there any answer to either question other than a repudiation of the question itself that wouldn't be considered a racist response?

Post reply on HN