Don't call "Nature Communications" "Nature". The prestige is totally different. Also, altmetrics aren't that relevant, maybe if you want to measure public hype.
Update: It seems the author read this and fixed it. Thanks!
Deep learning gets the glory, deep fact checking gets ignored
91–100 of 174 posts
Re: Deep learning gets the glory, deep fact checking gets ignored
#92Earlier quoted context omitted.
While that will not land him a Nobel prize, its miles ahead in terms of achievement and added value to mankind compared to most corporate employees. We wish we could say something like that about our past decade of work
People typically get paid as a thank you for their corporate work, not just a lukewarm 'thanks for pointing that out.'
A lot of the time, the work we do doesn’t get much recognition, and barely gets seen.But maybe it still helped in some small way.
Thinking about that makes it feel a little less disappointing.
Re: Deep learning gets the glory, deep fact checking gets ignored
#93Earlier quoted context omitted.
Almost nobody is "anti-science". The source of that labeling and division came from appeals to authority. You must do or believe this because it's "the science." If you don't, or you disagree, then you are anti-science. It has nothing to do with science, but rather people not finding that a sufficient justification for unpopular actions. For instance it's 100% certain that banning sugary drinks would dramatically imp…
> Almost nobody is "anti-science". Last I checked: - 15% of Americans don't believe in Climate Change[0] - 37% believe God created man in our current form within the last ~10k years (i.e. don't believe in evolution)[1] I don't think these are just rounding errors. They're large enough numbers that you should know multiple people who hold these beliefs unless you're in a strong bubble. I'm obviously with you in news a…
The stats you mention seem to suggest you are a believer in `The Science` - an anti scientific idea if ever there was one and one that's undergoing erosion day by day.
Re: Deep learning gets the glory, deep fact checking gets ignored
#94Earlier quoted context omitted.
> Almost nobody is "anti-science". Last I checked: - 15% of Americans don't believe in Climate Change[0] - 37% believe God created man in our current form within the last ~10k years (i.e. don't believe in evolution)[1] I don't think these are just rounding errors. They're large enough numbers that you should know multiple people who hold these beliefs unless you're in a strong bubble. I'm obviously with you in news a…
The stats you mention seem to suggest you are a believer in `The Science` - an anti scientific idea if ever there was one and one that's undergoing erosion day by day.
I'll bite, what's "The Science"
Me? I barely believe in the results of my experiments. But I also know what this poll is intending to ask and yeah, I read enough papers, processed enough data, did enough math, and tracked enough predictions that ended up coming true. That's enough to convince me it's pretty likely that those spending a fuck more time on it (and are the ones making those predictions that came true!) probably know what they're talking about.
Re: Deep learning gets the glory, deep fact checking gets ignored
#95Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.
You probably have to fight quite a few battles because many people know their papers aren't reproducible. More politics than science really.
It would be the biggest boon to science since sci-hub though.
And since a large set of studies won't be reproducible, you need human supervision as well, at least at first.
Re: Deep learning gets the glory, deep fact checking gets ignored
#96Man, I’ve been there. Tried throwing BERT at enzyme data once—looked fine in eval, totally flopped in the wild. Classic overfit-on-vibes scenario. Honestly, for straight-up classification? I’d pick SVM or logistic any day. Transformers are cool, but unless your data’s super clean, they just hallucinate confidently. Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest.…
> Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest.
If I gave a classroom of under grad students a multiple choice test where no answers were correct, I can almost guarantee almost all the tests would be filled out.
Should GPT and other LLMs refuse to take a test?
In my experience it will answer with the closest answer, even if none of the options are even remotely correct.
Re: Deep learning gets the glory, deep fact checking gets ignored
#97Man, I’ve been there. Tried throwing BERT at enzyme data once—looked fine in eval, totally flopped in the wild. Classic overfit-on-vibes scenario. Honestly, for straight-up classification? I’d pick SVM or logistic any day. Transformers are cool, but unless your data’s super clean, they just hallucinate confidently. Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest.…
>Lately, I just steal embeddings from big models and slap a dumb classifier on top. Works better, runs faster, less drama.
Sure but this is still indirectly using transformers.
Re: Deep learning gets the glory, deep fact checking gets ignored
#98Man, I’ve been there. Tried throwing BERT at enzyme data once—looked fine in eval, totally flopped in the wild. Classic overfit-on-vibes scenario. Honestly, for straight-up classification? I’d pick SVM or logistic any day. Transformers are cool, but unless your data’s super clean, they just hallucinate confidently. Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest.…
> Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest. If I gave a classroom of under grad students a multiple choice test where no answers were correct, I can almost guarantee almost all the tests would be filled out. Should GPT and other LLMs refuse to take a test? In my experience it will answer with the closest answer, even if none of the options are even remotely…
Not refuse, but remark that none of the answers seems correct. After all, we are only 2 days away from an AGI pro gamer researcher AI according to experts[1], so I would expect this behavior at least.
1: People who have a financial stake in the AI hype
Re: Deep learning gets the glory, deep fact checking gets ignored
#99Man, I’ve been there. Tried throwing BERT at enzyme data once—looked fine in eval, totally flopped in the wild. Classic overfit-on-vibes scenario. Honestly, for straight-up classification? I’d pick SVM or logistic any day. Transformers are cool, but unless your data’s super clean, they just hallucinate confidently. Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest.…
> Like giving GPT a multiple-choice test on gibberish—it will pick something, and say it with its chest. If I gave a classroom of under grad students a multiple choice test where no answers were correct, I can almost guarantee almost all the tests would be filled out. Should GPT and other LLMs refuse to take a test? In my experience it will answer with the closest answer, even if none of the options are even remotely…
I would love to see someone try this. I would guess 85-90% of undergrads would fill out the whole test, but not everyone. There are some people who believe the things that they know.
Re: Deep learning gets the glory, deep fact checking gets ignored
#100Before making AI do research, perhaps we should first let it __reproduce__ research. For example, give it a paper of some deep learning technique and make it produce an implementation of that paper. Before it can do that, I have no hope that it can produce novel ideas.
I thought you were going to say "give AI the first part of a paper (prompt) and let it finish it (completion)" as a validation AI can produce science at par with research results. Before it can do that, I have no hope that it can produce novel ideas.
I once had a university assignment where they provided the figures from a paper and we had to write the paper around the just the given figures.
A bit like how you might write a paper yourself - starting with the data.
As it turned out I thought the figures looked like data that might be from a paper referenced in a different lecturers set of lectures ( just on the conclusion, he hadn't shown the figures ) - so I went down the library ( this is in the days of non-digitized content - you had to physically walk the stacks ) and looked it up - found the original paper and then a follow up paper by the same authors....
I like to think I was just doing my background research properly.
I told a friend about the paper and before you know it the whole class knew - and I had to admit to the lecturer that I'd found the original paper when he wondered why the whole class had done so well.
Obviously this would be trivial today with an electronic search.