191–200 of 203 posts
i would like add some discussions. when i say "i", you can either read "i" as i am a pc or external reviewer, or maybe im a grad student. 1. i dont want say anything which gets me trouble, but hotcrp was set up wrong by the chair in that year ISCA. i know pc members were able to download the whole paper submission list. i personally saw _all_ papers and _all_ reviewer comments on anything i did not submit. it is like…
Earlier quoted context omitted.
is this true? Can you expound on this? which suspect papers are there? I know there are a lot of solid professors in UIUC like Joseph Torellas, Sarita Adve (C++ consistency models), Vikram Adve (who is the adviser of Chris Lattner - creator of LLVM). This is really bothersome and I hope it's not from one of those professors.
well let's take Torrellas. Check out the two papers at ISCA'20 with him as an author. Both use modified "cycle-accurate" simulators for the results. For now let's ignore all the issues with the accuracy of these simulators. Let's validate how "solid" the work is. Drop an e-mail to Torrellas and ask for the code they used so you can see if you can reproduce their work. Hopefully things have changed and they'll send yo…
I'm very curious - why you stress so much about publishing in that Elseviers, ACMEs and so on ?
It's unhealty.
First: there are so called "home pages" - instant publication ! I belive every univ have such infrastructure for their stuff, possible students too...
Not every paper is ground breaking discovery and not every paper should be published in international and GLOBAL "jurnal".
But if you realy insist then: let say USoA with 50 states so at least 50 universities... Then, if you want to publish collect, let say every half of the year, papers (eg. in Tex format) in your department, compile them into YourUniversityDepartmentYearNumber jurnal, print 50 of it, glue stamps and post to 50 libraries. Done.
Include cdrom.
Also you could do zines. With it's own domain name. Easy.
Good content will bring readers.
Do not forget about ToC, with urls.
But it's not recipe for all diseases you have there.
You need to step back and evaluate.
Also don't allow yourself to be morphed into asholes and in two generations you will be deans and proffesors.
And some environment are just viperies, move out of there !!!
Earlier quoted context omitted.
Not at all. The idea maybe was good (in long term with solid work). But none of our code were working, or works right now for that matter. Nothing works . And I am not saying they does not work as research codes tend to be ugly. I am saying that in literal sense of the word. It was totally wishful story glued together with beautiful sentences, published via a serious relationship in MICRO. There are sentences in that…
It does seem the prevailing writing style of academia is a big part of the problem. Papers a lot of time leave the reader feeling stupid. “Surely there’s wisdom here!?” the reader hopes. Maybe if papers taught the underlying concepts in a clearer, more transparent way we’d get better, more reproducible science
Today, it seems that using unnecessarily complex language is required to make work seem more impressive than it really is. Perhaps because much of today's published work is not that significant. I've been asked to rephrase perfectly clear and simple sentences solely to use fancier language and currently in vogue vernacular. None of that aids clarity or understanding.
Earlier quoted context omitted.
is this true? Can you expound on this? which suspect papers are there? I know there are a lot of solid professors in UIUC like Joseph Torellas, Sarita Adve (C++ consistency models), Vikram Adve (who is the adviser of Chris Lattner - creator of LLVM). This is really bothersome and I hope it's not from one of those professors.
well let's take Torrellas. Check out the two papers at ISCA'20 with him as an author. Both use modified "cycle-accurate" simulators for the results. For now let's ignore all the issues with the accuracy of these simulators. Let's validate how "solid" the work is. Drop an e-mail to Torrellas and ask for the code they used so you can see if you can reproduce their work. Hopefully things have changed and they'll send yo…
There are a lot of papers in ISCA that are based on cycle-accurate simulators. It has been like that since forever. How else would you evaluate new non-existent architectures that are bleeding-edge? FPGA? Most can't even afford that. Not to mention it's just not possible in a lot of cases. I agree with you that some work should be verifiable but your accusation is weak for this part. Maybe the community can push for verifiable work before getting published in ISCA since it is such a prestigious conference.
If you don't want cycle-accurate simulator based papers, you'd have to eliminate a large variety of body of work only possible with this approach. A lot of techniques in modern processors have seen their start in processor/system simulators and most of them are probably not cycle-accurate.
Earlier quoted context omitted.
is this true? Can you expound on this? which suspect papers are there? I know there are a lot of solid professors in UIUC like Joseph Torellas, Sarita Adve (C++ consistency models), Vikram Adve (who is the adviser of Chris Lattner - creator of LLVM). This is really bothersome and I hope it's not from one of those professors.
well let's take Torrellas. Check out the two papers at ISCA'20 with him as an author. Both use modified "cycle-accurate" simulators for the results. For now let's ignore all the issues with the accuracy of these simulators. Let's validate how "solid" the work is. Drop an e-mail to Torrellas and ask for the code they used so you can see if you can reproduce their work. Hopefully things have changed and they'll send yo…
2. i dont understand how you can pick on "cycle accurate" simulation when every simulator in our community has problems. at least in gem5 (one of torellas isca2020 paper), we as reviewers can look at the code. how about the famous "in house x86 pin based simulator"? most pin based simulators performance numbers should rightfully be joke, but we use them anyway, because we aren't going to rewrite them.
3. at the end of the day, most of our work is unverifiable, because we make so many approximations anyway. one young faculty told me "we just need to see the idea and determine if it makes sense". i just do not know if this is the right thing to do, or if its a lie we tell ourselves.
Earlier quoted context omitted.
it isn't always that the work is bad, it's just the review process for getting in has always seemed to be easier for people with the right connections. Not necessarily in an overt conspiratorial way like the recent allegations either. The reason people have latched on to the current set of allegations is there seems to be actual concrete proof of misconduct that can be acted on.
> it's just the review process for getting in has always seemed to be easier for people with the right connections What is this insinuation based on? Do you have data on how many papers by "people with the right connections" are rejected, for example?
i'm not sure this data is useful, because we'd disagree on "people with the right connections".
Earlier quoted context omitted.
Current U.S./Europe academia seems unable to change from within. It lives on government money which it itself controls and distributes to those who "play ball" - the others are pushed away. The only way a change can happen here if academia-independent people control and distribute money to individuals who showed talent/potential/results in the past. It will take massive outside interference to fix or supplant the cor…
That's not quite how it works and in my opinion not the main problem. Take for example DARPA grants, they are controlled by outside people and I don't think it works. You essentially have a "case worker" often with little understanding of the field pushing in some direction. Moreover the amount of admin around these grants is staggering. The root cause of the situation is that there is not enough money in the system…
Established elder scientists with little bias and little incentive to corrupt the country education/science system should be in charge, not administrative pencil pushers. Maybe hire some outsiders and set up some kind of independent commission of renowned scientists and budget planning/audit people to maintain oversight of the supported science groups.
But lack of money isn't really part of the problem, there is lot of money there and it often goes to waste. Putting in more isn't the solution - it would just amplify the current situation.
The fact there is competition for money isn't a bad thing. In theory it keeps people engaged and working efficiently. The problem with the current competition is that people are judged by their willingness to play ball, by flawed citation metrics, publicity, imaginary department points and relationships. Not by their originality and weight of their scientific contribution.
Earlier quoted context omitted.
well let's take Torrellas. Check out the two papers at ISCA'20 with him as an author. Both use modified "cycle-accurate" simulators for the results. For now let's ignore all the issues with the accuracy of these simulators. Let's validate how "solid" the work is. Drop an e-mail to Torrellas and ask for the code they used so you can see if you can reproduce their work. Hopefully things have changed and they'll send yo…
> ewers ever saw the code involved. Thi There are a lot of papers in ISCA that are based on cycle-accurate simulators. It has been like that since forever. How else would you evaluate new non-existent architectures that are bleeding-edge? FPGA? Most can't even afford that. Not to mention it's just not possible in a lot of cases. I agree with you that some work should be verifiable but your accusation is weak for this…
yes. Most "cycle-accurate" results are garbage. Do you show error bars on your results? Can you? Did you run on a variety of independently implemented simulators and show the results on all of them? Did you run the full reference inputs to SPEC all the way through? Why not?
The answer seems to be that it would be hard. But guess what, science is hard. Try complaining to a biologist sometime about your architecture paper that took so long to write, where you gathered all the results 2 weeks before the paper deadline.
It's fine if you come up with a new idea and run some simple proof-of-concept runs to show it might have merit, but don't pretend the results from an academic simulator hacked together by a sleep-deprived grad student have any real world merit.
Earlier quoted context omitted.
it isn't always that the work is bad, it's just the review process for getting in has always seemed to be easier for people with the right connections. Not necessarily in an overt conspiratorial way like the recent allegations either. The reason people have latched on to the current set of allegations is there seems to be actual concrete proof of misconduct that can be acted on.
> it's just the review process for getting in has always seemed to be easier for people with the right connections What is this insinuation based on? Do you have data on how many papers by "people with the right connections" are rejected, for example?
another anecdote, when first trying to raise awareness of the issue, when mentioning it to members of the community I never got the response "wow, how could this happen in computer architecture?" rather the response tended to be "wow, I can't believe it finally got bad enough that a student died"