131–140 of 160 posts
Earlier quoted context omitted.
> They chose to publish. So they are interested in seeing it reproduced or improved upon. Not necessarily, publishing also ensure that the stuff is no longer patentable.
Forgive me if I am wrong, but all of the techniques explored are already well known. So, what is going to be patented?
Worth pointing out here that in other scientific domains, papers routinely require hundreds of thousands of dollars, sometimes millions of dollars, of resources to produce. My wife works on high-throughout drug screens. They routinely use over $100,000 of consumables in a single screen, not counting the cost of the screening “libraries”, the cost of using some of the -$10mil of equipment in the lab for several weeks,…
I estimated that any paper that has mouse work and produced in a first world country (I.e. they have to do good by the animals), the minimum cost of that paper in expenses and salary would be $200,000. Average likely higher. Tens of thousands of papers a year published like this!
I wonder how many tons of CO2 that amounts to. Google Gemini estimated 125,000 tons of carbon emissions, but I don’t have the know-how to double check it.
If you use solar energy, then there is no CO2 emission. Right?
I think if you wanted to think about a big expense you'd look at AlphaStar.
The cost was probably the limiting factor.
Worth pointing out here that in other scientific domains, papers routinely require hundreds of thousands of dollars, sometimes millions of dollars, of resources to produce. My wife works on high-throughout drug screens. They routinely use over $100,000 of consumables in a single screen, not counting the cost of the screening “libraries”, the cost of using some of the -$10mil of equipment in the lab for several weeks,…
Earlier quoted context omitted.
I used to believe this line. But then I worked for a big tech company where my manager constantly made those remarks ("the difference in industry and academia is that in industry it has to actually work"). I then improved the generalization performance (i.e. "actually work") by over 100% and they decided not to update the model they were selling. Then again, I had a small fast model and it was 90% as accurate as the…
The post you are replying to is talking about high throughput assays for drug development. This is something actually run in a lab, not a model. As another person working at a biotech, I can assure you that screens are not just run as busy work.
While this is true, it is also true that the sentiment of this line is frequently used outside of the biotech (note that this entire website is primarily dominated by computer science posts). I think it is just worth mentioning that it is perfectly valid for discussions to not be pigeonholded and that you are perfectly allowed to talk about similarities in other fields (exact or inexact).
It is also true that my reply is not invalidated by changing settings. If you pay close attention you'll notice the generality of the comment outside of the specific example. In fact, this is exactly what the OP did, considering the article is about a machine learning paper and the example they used to __illustrate__ their point was about their wife's work in biotech. But the sentiment/point/purpose of their comment would not have changed were their wife to work in physics/engineering/chemistry/underwater basket weaving/whatever. So if this is your issue, I think they are misdirected and I ask that you please take it up with the OP and ensure that they know no comments in this thread may be about anything but ML papers by DeepMind. Illustrative examples out of domain are not allowed.
It is also true that this domain example was a product this company was selling. So I think you're being too quick to dismiss as not only did it run "in a lab" but the actual product runs in the real world. It has real customers who use the software.
It's also true I never accused anyone of doing "just [running] busy work" and that such an interpretation is grossly inaccurate. My final sentence should make this abundantly clear and I would argue is far more important in a setting such as medicine where I've said conveyed that you can sell ineffective or subpar products while still generating a profit and suggested that this probably shouldn't be the metric we care about (it certainly isn't what the spirit of those metrics are about).
But if you want to (implicitly) accuse me of derailing the conversation, I do not think you have the grounds to do so. But I will accuse you of doing so. If you disagree with my comment, you are more than welcome to reply in such a way. If you think my comment does not apply to dug discovery or think it only applies to ML[0], then you are welcome to state as much too and it is encouraged to state why. But the only derailing of the conversation has been the pigeonholing you have applied. If you think this is wrong, I still welcome a response to that as I am happy to learn how to communicate better but you did also catch me on a day where I'm not happy to be unreasonably and willfully misinterpreted.
[0] I didn't tell you the application... a bit presumptuous are we?
Earlier quoted context omitted.
Well, to be sure, mouse research consistently produces amazing cures for cancer, insomnia, lost limbs, and even gravity itself. Sure, none of it translates to humans, but it's an important source of headlines for high impact journals and science columnists.
This is also true for machine learning papers. They cure cancer, discover physics, and all sorts of things. Sure, they don't actually translate to useful science, but they are highly valuable pieces of advertisements. And hey, maybe someday they might!
Earlier quoted context omitted.
The post you are replying to is talking about high throughput assays for drug development. This is something actually run in a lab, not a model. As another person working at a biotech, I can assure you that screens are not just run as busy work.
No they’re not busywork, but not all such screens are directly in the drug discovery pipeline.
Earlier quoted context omitted.
This is also true for machine learning papers. They cure cancer, discover physics, and all sorts of things. Sure, they don't actually translate to useful science, but they are highly valuable pieces of advertisements. And hey, maybe someday they might!
AlphaFold has basically paid the bills for a decade's worth of machine learning research. It's been that transformative.
My comment was a bit tongue in cheek. Not every research is going to be profitable or eventually profitable, but that also doesn't mean it isn't useful. If we're willing to account for the indirect profits via learning what doesn't work (an important part of science), then this vastly diminishes the number of worthless papers (to essentially those that are fraudulent or plagiarism)
But as specifically for AlphaFold, I'm going to need a citation on that. If I understand the calculus correctly, Google acquired DM in 2014 for somewhere between $525 million to $850 million, and yearly spends a similar amount each year along with forgiving a 1.5bn dollar debt[0]. So I think (VERY) conservatively we can say $2bn (I think even $4bn is likely conservative here)? While I see articles that discuss how the value could be north of $100bn[1] (which certainly surpasses a very liberal estimate of costs), I have yet to see evidence that this is actual value that has been returned to Google. I can only find information about 2022 and 2023 having profits in the ballpark of $60m.
This isn't to say that AlphaFold won't offset all the costs (I actually believe it will), but I your sentence does not suggest speculation but rather actualization ("has basically paid", "been"). I think that difference matters enough that we have a dozen cliches with similar sentiment. In the same way my annoyance is not that we are investing in ML[2], but how quick we are to make promises and celebrate success[3]. Actually, my concern is that while hype is necessary, overdoing it allows charlatans[4] to more easily enter the space. And if they are able to gain a significant foothold (I believe that is happening/has happened) then this is actually destructive to those who actually wish to push forward technology.
[0] https://www.quora.com/How-much-money-did-Google-spend-on-Dee...
[1] https://www.bloomberg.com/news/articles/2024-05-08/deepmind-...
[2] disclosure, I'm an ML researcher. I actually am in favor of more funding. Though different allocation.
[3] I'm willing to concede that success is realistically determined by how one measures success, and that this may be arbitrary and no objective measure actually exists or is possible.
[4] One needs not knowingly be a charlatan. Only that the claims made are false or inaccurate. There are many charlatans who believe in the snake oil they sell. Most of these are unwilling to acknowledge critiques. A clear example is religion. If you believe in a religion, this applies to all religious organizations except the one you are a part of. If you are not religious, well the same sentence holds true but the resultant set is one larger.
3USD/hour on the H100 is much more expensive than a reasonable amortized full ownership cost, unless one assumes the GPU is useless within 18 months, which I find a bit dramatic. The MFU can be above 40% and certainly well above the 35% in the estimate, also for small models with plain pytorch and trivial tuning [1] I didnt read the linked paper carefully but I seriously doubt the google team used vocab embedding lay…
Do you have sources for "The MFU can be above 40% and certainly well above the 35 % in the estimate"? Looking at [1], the authors there claim that their improvements were needed to push BERT training beyond 30% MFU, and that the "default" training only reaches 10%. Certainly numbers don't translate exactly, it might well be that with a different stack, model, etc., it is easier to surpass, but 35% doesn't seem like a…