This is so crazy good, someone needs to do a Turing test by sending it to some unsuspecting publishers. I get the feeling that debatepocolypse is not far away. Every forum can now be spammed with reasonable sounding gibberish that humans will have to slog through.
Better Language Models and Their Implications
121–130 of 138 posts
Re: Better Language Models and Their Implications
#122Earlier quoted context omitted.
> Agreed that "simply" scaling up with more compute will result in progress and useful systems, and work in that direction is interesting and valuable. But, while we may not need new architectures or training objectives to make progress, we do need them to approach human level sample complexity. Yes, agreed. Nothing I said above contradicts that! :-) > Humans don't need to read through 40 GB of text multiple times to…
Conscious processes are estimated to work on the order of 10^2 bits. Vision, at the retina, is estimated at 10^7 bits/sec. It drops another order of magnitude by V1. Also note that long as they're not isolated, a deaf and blind person has no trouble getting to full human reasoning ability despite being vastly more impoverished in available data compared to the average person. A human will also be learning vision, hea…
How do you estimate the effective (digital) bitrate of an inherently analogue system?
Re: Better Language Models and Their Implications
#123I’m most impressed by its ability to answer questions about the text. Why can’t someone built something like this on top of Wikipedia? That would be amazing to ask Wikipedia any question you can think of.
http://www.seobythesea.com/2014/10/google-fact-questions-ent...
Re: Better Language Models and Their Implications
#124This is so crazy good, someone needs to do a Turing test by sending it to some unsuspecting publishers. I get the feeling that debatepocolypse is not far away. Every forum can now be spammed with reasonable sounding gibberish that humans will have to slog through.
That's not a Turing test. A Turing test is an ongoing dialog, not a one-shot filter.
Re: Better Language Models and Their Implications
#125Earlier quoted context omitted.
That's not a Turing test. A Turing test is an ongoing dialog, not a one-shot filter.
Add some bias to create dialogues when sampling based on movie subtitles and I could see it passing proper Turing test.
Re: Better Language Models and Their Implications
#126Earlier quoted context omitted.
Add some bias to create dialogues when sampling based on movie subtitles and I could see it passing proper Turing test.
A proper Turing test allows you to ask questions whose answers must indicate novel introspection, learning, and social awareness. It's absurd to think any computer system today would pass such a test.
Re: Better Language Models and Their Implications
#127This tech can easily be used to flood humanity’s shared brain with auto-generated propaganda. Schizophrenia of the internet in a way. There is plenty of incentive with Google algorithms favoring number of words and relevant keywords in content for rankings - you could have NLP bots lifting junk sites to top results. To step ahead in that chess game, a detection tool for fake would be just training grounds for better…
Web of trust. Your results and how much you trust given text is based on who gave it to you. You assign trust to people you know, then it's a small world effect and recursive trust calculation based on who they trust. Centralization won't work. Whether something is good or bad or fake must be subjective and based on your personal network / your beliefs. Otherwise, long term, I don't see how you could avoid dystopia.
If dissenting opinions are never allowed into your sphere of influence then we may continue to see an accelerating polarization of our society towards extremes.
Re: Better Language Models and Their Implications
#128Re: Better Language Models and Their Implications
#129Earlier quoted context omitted.
Web of trust. Your results and how much you trust given text is based on who gave it to you. You assign trust to people you know, then it's a small world effect and recursive trust calculation based on who they trust. Centralization won't work. Whether something is good or bad or fake must be subjective and based on your personal network / your beliefs. Otherwise, long term, I don't see how you could avoid dystopia.
This has a dangerous implication that we've seen recently with the newsfeed "bubble." If dissenting opinions are never allowed into your sphere of influence then we may continue to see an accelerating polarization of our society towards extremes.
I know the more popular bubble is information one and you make a fair point, but I believe it's powered more by recommendation systems than the things that you trust.
I got distracted lately and I wanted to clean it up before putting it out there but if somebody is bored: http://comboy.pl/wot.html
Re: Better Language Models and Their Implications
#130Earlier quoted context omitted.
Add some bias to create dialogues when sampling based on movie subtitles and I could see it passing proper Turing test.
A proper Turing test allows you to ask questions whose answers must indicate novel introspection, learning, and social awareness. It's absurd to think any computer system today would pass such a test.
Feel free to predict and comment too!