Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alias-Free GAN

nvlabs.github.io

Post reply on HN
Print view View original

31–40 of 83 posts

Re: Alias-Free GAN

#31

Post by aaron-santos »

sillysaurusx wrote: post #23
aaron-santos wrote: post #22

Earlier quoted context omitted.

Thank you for calling this out. It's critically important that people understand the difference between model, code, and paper and what they mean. It's also important that people understand that even if code is provided, it's commercially useless. From the NVAE license as an example[1] > The Work and any derivative works thereof only may be used or intended for use non-commercially. It's a great example of the differ…

Unfortunately, I must call you out too, my friend. With love. Because it’s crucially important that we protect the scientific method here. The sole goal is to help people like me reproduce the model. If I can’t reproduce the model, I can’t verify the paper. When I saw “commercial” and then “open source” in your comment, I said “oh no…” My duty is to the scientific method, so I don’t care if it’s the most restrictive…

That's fine, at least you're open about the knowledge for knowledge's sake position. There's more than one way to judge something.

Re: Alias-Free GAN

#32

Post by evo »

I wonder if there are learnings from this that could be transposed into the 1-D domain for audio; as far as I know, aliasing is a frequent challenge when using deep learning methods for audio (e.g. simulating non-linear circuits for guitar amps).

Re: Alias-Free GAN

#33

Post by phreeza »

sillysaurusx wrote: post #27
phreeza wrote: post #26

Earlier quoted context omitted.

Expecting fully executable code to accompany every publication is kind of unique to the modern ML research Scene. As someone from a very different computational research field, where zero code is the norm, not the exception, this reads as a somewhat entitled rant. Reimplementation of a paper is actually a test of the robustness of the results. If you download the code of a previous paper, there may be some assumption…

I wasn't sure whether to post my edit as a separate comment or not, but I significantly expanded my comment just now, that helps explain my position. I'd be very interested in your thoughts on that position, because if it's mistaken, I shouldn't be saying it. It represents whatever small contribution I can make to fellow new ML researchers, which is roughly: "watch out. " In short, for two years, I kept trying to imp…

I get your frustrations with this state of affairs, but for the reasons I mentioned above, I don't think providing the model and code is a panacea here. Maybe the last few years have also set an unrealistic expectation for the pace of progress. In my (former) field of theoretical neuroscience, if a paper was not reproducible, this knowledge kind of slowly diffused through the community, mostly through informal conversations with people who tried to reproduce or extend a given approach. But this takes several years, not the kind of timescale that modern ML research operates on.

Fwiw I think actual knowledge is there in the ML literature, but it's not in these Benchmark-chasing highly tuned papers. It's more high level stuff, like basic architecture building blocks etc. GANs and Transformers for example. They undeniably work, and the knowledge needed to implement them can probably be conveyed in a few pages maximum. No need for an implementation to be provided by the author, really.

Re: Alias-Free GAN

#34

Post by sillysaurusx »

msapaydin wrote: post #29
sillysaurusx wrote: post #27

Earlier quoted context omitted.

I wasn't sure whether to post my edit as a separate comment or not, but I significantly expanded my comment just now, that helps explain my position. I'd be very interested in your thoughts on that position, because if it's mistaken, I shouldn't be saying it. It represents whatever small contribution I can make to fellow new ML researchers, which is roughly: "watch out. " In short, for two years, I kept trying to imp…

I think that not being able to reproduce the results claimed in a paper is not specific to ML research. While working as a post-doc at a top university research lab, i spent years trying to understand how it can be that some software that was supposed to corresponds to the well cited paper did not even come close to reproducing the results of the said paper, and that the primary author went on to become a prof at a t…

Thank you!!

i spent years trying to understand how it can be that some software that was supposed to corresponds to the well cited paper did not even come close to reproducing the results of the said paper,

This was my exact experience. I didn’t understand why I kept having it, and kept blaming myself for not being careful enough. My code must be wrong, or the data, or something.

Nah. It was the idea.

Kept feeling like a kick in the gut, until here we are today, when I’m warning everyone that Karras, of all people, might publish such a thing.

I really appreciate that you posted this, because I’m so happy I wasn’t alone in the feeling of “what’s going on, here…?”

Re: Alias-Free GAN

#35

Post by xksteven »

phreeza wrote: post #26
sillysaurusx wrote: post #25

Earlier quoted context omitted.

There is no reason to doubt the open-source release of this work will be any different. Then this is not a scientific contribution yet. We must wait and see. The most important tenet of science, is to doubt. I didn’t even read the name on the paper before I wrote my comment. Yes, I know this group. They’re why I got into ML, along with the group from OpenAI who published GPT-2. Because A+ science. Their claims here a…

Expecting fully executable code to accompany every publication is kind of unique to the modern ML research Scene. As someone from a very different computational research field, where zero code is the norm, not the exception, this reads as a somewhat entitled rant. Reimplementation of a paper is actually a test of the robustness of the results. If you download the code of a previous paper, there may be some assumption…

Sometimes reimplantation is impossible without the code and the paper goes on to win awards because it's by a famous scientist. Then if the reimplantation doesn't work most of the time the graduate students are blamed instead of the original work.

There are always assumptions. At least with public code and models those assumptions are laid bare for all to see and potentially expose any bad assumptions.

Re: Alias-Free GAN

#36

Post by nerdponx »

phreeza wrote: post #26
sillysaurusx wrote: post #25

Earlier quoted context omitted.

There is no reason to doubt the open-source release of this work will be any different. Then this is not a scientific contribution yet. We must wait and see. The most important tenet of science, is to doubt. I didn’t even read the name on the paper before I wrote my comment. Yes, I know this group. They’re why I got into ML, along with the group from OpenAI who published GPT-2. Because A+ science. Their claims here a…

Expecting fully executable code to accompany every publication is kind of unique to the modern ML research Scene. As someone from a very different computational research field, where zero code is the norm, not the exception, this reads as a somewhat entitled rant. Reimplementation of a paper is actually a test of the robustness of the results. If you download the code of a previous paper, there may be some assumption…

It's not an entitled rant, other fields just have dismally low standards.

Re: Alias-Free GAN

#37

Post by sillysaurusx »

minimaxir wrote: post #28
sillysaurusx wrote: post #21

Earlier quoted context omitted.

This isn't true. I do ML every day. You are mistaken. I click the website. I search "model". I see two results. Oh no, that means no download link to model. I go to the github. Maybe model download link is there. I see zero code : https://github.com/NVlabs/alias-free-gan Zero code. Zero model. You, and everyone like you, who are gushing with praise and hypnotized by pretty images and a nice-looking pdf, are doing dam…

The repo says the code will be available in September: that's a reasonable timeframe for the necessary polish/legal clearance.

(Agreed, fwiw. What’s going on here isn’t a criticism of this work specifically, but the trend of everyone thinking that this is science generally. For example, it’s true the code is coming in September. And, you and I both know it’s probably gonna have a model release, just because it’s more impressive, big-name Karras nVidia work. But it might not have a model release. I give that at least 40% odds. If it doesn’t, then everything I said above will be true about that too, Karras or not. People keep doing that, and we have to call out that this is approximately useless for you and me. Actually, I was going to say maybe it’s useful for you, but you’re the language model hacker and I’m the GAN hacker, and I assure you, code alone is useless for me. If it’s useful to you, I would love to know how it helps you verify the scientific method.)

Re: Alias-Free GAN

#38

Post by skybrian »

phreeza wrote: post #33
sillysaurusx wrote: post #27

Earlier quoted context omitted.

I wasn't sure whether to post my edit as a separate comment or not, but I significantly expanded my comment just now, that helps explain my position. I'd be very interested in your thoughts on that position, because if it's mistaken, I shouldn't be saying it. It represents whatever small contribution I can make to fellow new ML researchers, which is roughly: "watch out. " In short, for two years, I kept trying to imp…

I get your frustrations with this state of affairs, but for the reasons I mentioned above, I don't think providing the model and code is a panacea here. Maybe the last few years have also set an unrealistic expectation for the pace of progress. In my (former) field of theoretical neuroscience, if a paper was not reproducible, this knowledge kind of slowly diffused through the community, mostly through informal conver…

I have no particular expertise here, but I wonder if you've learned to accept a mostly-broken process? We have the Internet, so why settle for slow diffusion over years instead of rapid communication?

Why should graduate students have to spend years trying to reproduce stuff that turns out to be no good? Nobody should have to put up with getting their time wasted like that.

Re: Alias-Free GAN

#39

Post by gonehome »

sillysaurusx wrote: post #34
msapaydin wrote: post #29

Earlier quoted context omitted.

I think that not being able to reproduce the results claimed in a paper is not specific to ML research. While working as a post-doc at a top university research lab, i spent years trying to understand how it can be that some software that was supposed to corresponds to the well cited paper did not even come close to reproducing the results of the said paper, and that the primary author went on to become a prof at a t…

Thank you!! i spent years trying to understand how it can be that some software that was supposed to corresponds to the well cited paper did not even come close to reproducing the results of the said paper, This was my exact experience. I didn’t understand why I kept having it, and kept blaming myself for not being careful enough. My code must be wrong, or the data, or something . Nah. It was the idea. Kept feeling l…

That seems like a worthwhile thing to publicize in and of itself?

The replication crisis in psychology threw out 50% or so of supposed scientific results.

If this (or just straight fraud) is common elsewhere, it seems like knowing about that would be a good thing for science.

Re: Alias-Free GAN

#40

Post by varispeed »

sillysaurusx wrote: post #21
ansk wrote: post #12

This group of researchers consistently demonstrates a degree of empirical rigor that is unmatched across any other ML lab in industry or academia - remarkable empirical results as always, reproducible experiments, open-source and well-engineered codebase, and valuable insights about low-level learning dynamics and high-level emergent artifacts. Applied ML wouldn't have such a bad rap if more researchers held themselv…

This isn't true. I do ML every day. You are mistaken. I click the website. I search "model". I see two results. Oh no, that means no download link to model. I go to the github. Maybe model download link is there. I see zero code : https://github.com/NVlabs/alias-free-gan Zero code. Zero model. You, and everyone like you, who are gushing with praise and hypnotized by pretty images and a nice-looking pdf, are doing dam…

I am not into ML, but from time to time I like to look how this is made and remember only once seeing the code and a model, which I thought was exception from the "norm". Good that more people are calling this out!
Post reply on HN