Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

ai.googleblog.com

Post reply on HN
Print view View original

21–30 of 36 posts

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#21

Post by SanderNL »

nl wrote: post #20
frisco wrote: post #2

Is there anything actually released here? Just a paper? No weights, not even code ?! No interactive product of any kind? The degree of Google’s inability to actually ship anything even now is totally mindblowing

This is absolutly wrong. Google (and other large industry labs) have the budget and resources to experiment. They should be encouraged and thanked for publishing what works. It's not super hard to reimplement a published paper. But running multiple experiments is hard for individuals or smaller companies.

> thanked for publishing what they and they alone claim works.

Fixed.

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#22

Post by throwaway29303 »

frisco wrote: post #2

Is there anything actually released here? Just a paper? No weights, not even code ?! No interactive product of any kind? The degree of Google’s inability to actually ship anything even now is totally mindblowing

You should read this https://news.ycombinator.com/item?id=35824408

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#23

Post by anentropic »

reaperman wrote: post #14
fpgaminer wrote: post #7

Earlier quoted context omitted.

A research paper by itself isn't worth nothing, sure, but without the ability to reproduce the paper or even check their results it's ... not worth much.

The architecture is described enough to re-implement it and train it on known datasets/benchmarks such as VQA2. A single man with a medical degree named Phil Wang ('lucidrains')[0] has the ability to reproduce most of these papers by himself . He has 246 GitHub repos[1], most of which appear to be reproductions of models which are only described in papers that had no associated code or models released, such as [2]. O…

Given that fact, why don't the paper authors just release the artefacts then?

If it's supposed to stay secret, what's the point of "here's instructions for how to reproduce our big secret"?

Presumably the societal purpose of papers is to share knowledge, and the individual purpose is to take credit and win prestige.

It seems like the first purpose would be better served by also publishing code etc, and the second purpose wouldn't be harmed by it?

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#24

Post by nl »

SanderNL wrote: post #21
nl wrote: post #20

Earlier quoted context omitted.

This is absolutly wrong. Google (and other large industry labs) have the budget and resources to experiment. They should be encouraged and thanked for publishing what works. It's not super hard to reimplement a published paper. But running multiple experiments is hard for individuals or smaller companies.

> thanked for publishing what they and they alone claim works. Fixed.

I'm not aware of any cases where this isn't the case and I personally have implemented a number of their papers.

Are you aware of other cases?

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#25

Post by mirker »

anentropic wrote: post #23
reaperman wrote: post #14

Earlier quoted context omitted.

The architecture is described enough to re-implement it and train it on known datasets/benchmarks such as VQA2. A single man with a medical degree named Phil Wang ('lucidrains')[0] has the ability to reproduce most of these papers by himself . He has 246 GitHub repos[1], most of which appear to be reproductions of models which are only described in papers that had no associated code or models released, such as [2]. O…

Given that fact, why don't the paper authors just release the artefacts then? If it's supposed to stay secret, what's the point of "here's instructions for how to reproduce our big secret"? Presumably the societal purpose of papers is to share knowledge, and the individual purpose is to take credit and win prestige. It seems like the first purpose would be better served by also publishing code etc, and the second pur…

Because the authors don’t get a large reward for open sourcing the work and they stand to lose future value by lowering the gate to competition. You may want the code, but Google will not care (or it might dislike it).

Look at GPT-3+, OpenAI gets fame and fortune while people struggle to reproduce their last-gen models.

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#26

Post by mirker »

light_hue_1 wrote: post #3
frisco wrote: post #2

Is there anything actually released here? Just a paper? No weights, not even code ?! No interactive product of any kind? The degree of Google’s inability to actually ship anything even now is totally mindblowing

That's Google. I don't bother to read most Google papers unless someone tells me that they're doing something astounding. Just because I know I don't have access to their models, their code or their data. So what's the point? As a community we need to stop accepting and stop citing papers like these. There is no science without replicability, and it is literally impossible to replicate this work. It's not worth the p…

There is a ton of value. OpenAI having proprietary LLMs single handedly pivoted the entire field to LLMs. A random GitHub repository doesn’t come close to impact.

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#27

Post by sitkack »

m00x wrote: post #19

Does anyone know the performance of this vs BLIP-2?

Given the exposure hn has had to blip-2 it might be good to provide more context.

https://arxiv.org/abs/2301.12597

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

Were you expecting a response?

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#28

Post by sitkack »

alsodumb wrote: post #12
fpgaminer wrote: post #10

Earlier quoted context omitted.

Conferences _should_ ban papers that don't release code or other means of reliable reproduction. The only reason they don't is because "research" in ML has more or less been a joke compared to any other established scientific field. And I'm not going to give Google the benefit of the doubt. At the very least I'll treat them like any random stranger publishing a paper. But in reality I treat their papers with a heavy…

Funny you compare ML with other fields - in my experience ML is the most open and reproducible area of scientific research, by far. Talk to researchers in other areas, many write “dataset and code available by request” but never share it, have custom CFS solvers and write papers with it but never release the code, and do experiments and leave out all details in the paper making it impossible to reproduce. You are fre…

I think a large part of the innovation in ML research is precisely because the code is release. The prevalence of a github.io page with the code, the paper, slides and a video presentation is amazing. I would love to see this practice extended for every other paper in every domain.

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#29

Post by xiphias2 »

anentropic wrote: post #23
reaperman wrote: post #14

Earlier quoted context omitted.

The architecture is described enough to re-implement it and train it on known datasets/benchmarks such as VQA2. A single man with a medical degree named Phil Wang ('lucidrains')[0] has the ability to reproduce most of these papers by himself . He has 246 GitHub repos[1], most of which appear to be reproductions of models which are only described in papers that had no associated code or models released, such as [2]. O…

Given that fact, why don't the paper authors just release the artefacts then? If it's supposed to stay secret, what's the point of "here's instructions for how to reproduce our big secret"? Presumably the societal purpose of papers is to share knowledge, and the individual purpose is to take credit and win prestige. It seems like the first purpose would be better served by also publishing code etc, and the second pur…

Probably because the research code is not as nice as one rewritten from scratch anyways, and it's using internal data sets / APIs.

They just want to get onto the next research instead of taking time to publish a clean open-source implementation, which can (and will) be done by somebody else anyways.

Re: MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks

#30

Post by m00x »

sitkack wrote: post #27
m00x wrote: post #19

Does anyone know the performance of this vs BLIP-2?

Given the exposure hn has had to blip-2 it might be good to provide more context. https://arxiv.org/abs/2301.12597 https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... Were you expecting a response?

I'm not sure what you mean. HN has a wide array of people that have multiple sources of information. I was throwing a question out there hoping an expert in the matter would know.

They're both multi-modal models for zero-shot answering.

I'm not sure what the purpose of this comment is. If no experts see this, I won't get an answer. You don't need to reply with something completely useless that adds nothing to the topic.

Post reply on HN