Gradients are not all you need
arxiv.org
Gradients are not all you need
1–10 of 71 posts
Re: Gradients are not all you need
#2My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
Re: Gradients are not all you need
#3My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
I'd like to imagine quirky titles make them harder for other researchers to find, leading to lower citation counts, but that may be wishful thinking.
Re: Gradients are not all you need
#4My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
There is a lot of jargon in ML, an example is found in the object detection literature where you will often find sentences like such: "we present a novel simple non-hierarchical feature network backbone with window-shifted attention based learning accommodating neck+regression head fine-tuning or masked cascade RCNN second stage." I'm half joking. Surveys are often a godsend.
Re: Gradients are not all you need
#5My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
I'd like to imagine quirky titles make them harder for other researchers to find, leading to lower citation counts, but that may be wishful thinking.
I think this trend is happening because of the entire buzz around OPENAI LLMs and that 2017 Google Research Paper
Re: Gradients are not all you need
#6Earlier quoted context omitted.
I'd like to imagine quirky titles make them harder for other researchers to find, leading to lower citation counts, but that may be wishful thinking.
I think this trend is happening because of the entire buzz around OPENAI LLMs and that 2017 Google Research Paper
Ironically, attention is what you need to get high citation counts
Whether quirky titles help or hurt with that i think depends on name recognition and the publication venue
Re: Gradients are not all you need
#7My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
> "Understanding Limitations and Chaos-Based Failures in Gradient-Based Optimization Methods." (gpt4)
Fixed with ML.
Re: Gradients are not all you need
#8My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
The clever titles are more brandable. See the citations for "Attention is all you need" or "Chinchilla limit" versus more mundane titles.
Re: Gradients are not all you need
#9My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
Remember the days after Rowhammer and Heartbleed, when every new security vulnerability needed its own catchy name and domain name website? This is the science version of that.
Branding is eating the world.
Re: Gradients are not all you need
#10My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain 'evolutionary pressure' where clever titles 'outcompete' dryer, more descriptive titles (or it seems that way). But i don't like it.
The clever titles are more brandable. See the citations for "Attention is all you need" or "Chinchilla limit" versus more mundane titles.
I'm pretty sure Attention is all you need would have gotten a lot of citations even if it had a "boring" title. It was a groundbreaking paper with lots of good ideas.