Robot-written reviews fool academics • Hacker News Board

Re: Robot-written reviews fool academics

#21

Post by more_original » Fri, Sep 23, 2016, 7:49 PM UTC

GuiA wrote: post #17

more_original wrote: post #16

Earlier quoted context omitted.

ACM has a wide range of conferences. There are conferences with an A* ranking, but also ones with a C ranking. Submissions to A*-conferences are not 80% garbage. http://portal.core.edu.au/conf-ranks/?search=acm&by=all

Your link shows that over 75% of the conferences are B or lower. So sure, the submissions to A* conferences are not 80% garbage, but these conferences are 4% of the total. Not really representative of academic work as a whole.

Sure, I was just giving a possible reason for the different experiences of reviewing for ACM conferences.

Re: Robot-written reviews fool academics

#22

Post by eli_gottlieb » Fri, Sep 23, 2016, 9:42 PM UTC

GuiA wrote: post #17

more_original wrote: post #16

Earlier quoted context omitted.

ACM has a wide range of conferences. There are conferences with an A* ranking, but also ones with a C ranking. Submissions to A*-conferences are not 80% garbage. http://portal.core.edu.au/conf-ranks/?search=acm&by=all

Your link shows that over 75% of the conferences are B or lower. So sure, the submissions to A* conferences are not 80% garbage, but these conferences are 4% of the total. Not really representative of academic work as a whole.

Oh hey, cool! My paper writing up my MSc thesis was at a B-grade conference with a 26% acceptance rate. That's actually more selective than I expected, given my opinion of my own work.

Re: Robot-written reviews fool academics

#23

Post by eli_gottlieb » Fri, Sep 23, 2016, 9:43 PM UTC

qntty wrote: post #18

jfroma wrote: post #5

It reminds me the "Postmodernism Generator" http://www.elsewhere.org/pomo/

and similar projects for cs and math papers http://thatsmathematics.com/mathgen/ https://pdos.csail.mit.edu/archive/scigen/

And of course http://www.theproofistrivial.com/

Re: Robot-written reviews fool academics

#24

Post by digi_owl » Fri, Sep 23, 2016, 10:20 PM UTC

CobrastanJorji wrote: post #10

If you generated papers with MIT's SCIgen, reviewed them with this, and then responded to those reviews with short notes from Cleverbot, you could put together a whole collection of "peer-reviewed", fee-charging journals. You'd want to make sure they cross-cited each other heavily, thus earning the journals and their "authors" prestige.

Now to set this up to discredit the big issues of the day...

Re: Robot-written reviews fool academics

#25

Post by thaumasiotes » Sat, Sep 24, 2016, 5:51 AM UTC

SixSigma wrote: post #14

"Markov chains still produce credible sentences after 30 years" reports Mark V Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney

But Markov chains have never produced credible sentences. They produce crazy schizophrenic sentences that begin talking about one thing and end up talking about something entirely different because they went on farther than their lookback. Just look at the example in your link:

> It looks like Reagan is holding back the arms of the American eating public have changed dramatically, and it got pretty boring after about 300 games.

That's not going to get past anyone who reads it in any context. The method here was to copy entire well-formed sentences from existing reviews.

Re: Robot-written reviews fool academics

#26

Post by [deleted] » Sat, Sep 24, 2016, 1:54 PM UTC

thaumasiotes wrote: post #25

SixSigma wrote: post #14

"Markov chains still produce credible sentences after 30 years" reports Mark V Shaney https://en.wikipedia.org/wiki/Mark_V._Shaney

But Markov chains have never produced credible sentences. They produce crazy schizophrenic sentences that begin talking about one thing and end up talking about something entirely different because they went on farther than their lookback. Just look at the example in your link: > It looks like Reagan is holding back the arms of the American eating public have changed dramatically, and it got pretty boring after about…

[deleted]