Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interview with gwern

dwarkeshpatel.com

Post reply on HN
Print view View original

101–110 of 262 posts

Re: Interview with gwern

#101

Post by YeGoblynQueenne »

nocobot wrote: post #95
aubanel wrote: post #90

Earlier quoted context omitted.

> For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS".

is it really? this is the most common example for context free languages and something most first year CS students will be familiar with. totally agree that you can be a great engineer and not be familiar with it, but seems weird for an expert in the field to confidently make wrong statements about this.

Thanks, that's what I meant. a^nb^n is a standard test of learnability.

That stuff is still absolutely relevant, btw. Some DL people like to dismiss it as irrelevant but that's just because they lack the background to appreciate why it matters. Also: the arrogance of youth (hey I've already been a postdoc for a year, I'm ancient). Here's a recent paper on Neural Networks and the Chomsky Hierarchy that tests RNNs and Transformers on formal languages (I think it doesn't test on a^nb^n directly but tests similar a-b based CF languages):

https://arxiv.org/abs/2207.02098

And btw that's a good paper. Probably one of the most satisfying DL papers I've read in recent years. You know when you read a paper and you get this feeling of satiation, like "aaah, that hit the spot"? That's the kind of paper.

Re: Interview with gwern

#102

Post by okasaki »

Interesting that there's no mention of human biodiversity (aka blacks are dumb), as if you spend five minutes on #lesswrong you'll notice that that's a big issue for gwern and the other goons.

Re: Interview with gwern

#103

Post by tim333 »

madcaptenor wrote: post #3
rahidz wrote: post #2

Their voice sounds so realistic, that unlike the avatar, my monkey brain is not fathoming it being unreal.

From https://www.dwarkeshpatel.com/p/gwern-branwen : "In order to protect Gwern's anonymity, I proposed interviewing him in person, and having my friend Chris Painter voice over his words after. This amused him enough that he agreed."

Thanks. I was kind of puzzled that the voice is so natural and the avatar so freaky.

Re: Interview with gwern

#104

Post by newmanpo »

YeGoblynQueenne wrote: post #84

This will come across as vituperative and I guess it is a bit but I've interacted with Gwern on this forum and the interaction that has stuck to me is in this thread, where Gwern mistakes a^nb^n as a regular (but not context-free) language (and calls my comment "not even wrong"): https://news.ycombinator.com/item?id=21559620 Again I'm sorry for the negativity, but already at the time Gwern was held up by a certain, l…

I take the Feynman view here; vain memory tricks are not themselves net new production, so just look known things up in the book.

Appreciate the diversity in the effort, but engineering is making things people can use without having to know it all. Far more interesting endeavor than being a human Google search engine.

Re: Interview with gwern

#106

Post by tim333 »

demaga wrote: post #78

> I love the example of Isaac Newton looking at the rates of progress in Newton's time and going, “Wow, there's something strange here. Stuff is being invented now. We're making progress. How is that possible?” And then coming up with the answer, “Well, progress is possible now because civilization gets destroyed every couple of thousand years, and all we're doing is we're rediscovering the old stuff.” The link in th…

I think people click upvote on reading an interesting title / first couple of lines.

Then there isn't a downvote option if it proves poor.

Re: Interview with gwern

#107

Post by YeGoblynQueenne »

aubanel wrote: post #90
YeGoblynQueenne wrote: post #84

This will come across as vituperative and I guess it is a bit but I've interacted with Gwern on this forum and the interaction that has stuck to me is in this thread, where Gwern mistakes a^nb^n as a regular (but not context-free) language (and calls my comment "not even wrong"): https://news.ycombinator.com/item?id=21559620 Again I'm sorry for the negativity, but already at the time Gwern was held up by a certain, l…

> For me that's just a great example of how basically the vast majority of AI influencers (who vie for influence on social media, rather than research) are basically clueless about AI and CS This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS".

>> This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS".

I think, engineers, yes, especially those who don't have a background in academic CS. But scientists, no, I don't think so. I don't think it's possible to be a computer scientist without knowing the difference between a regular and a super-regular language. As to knowing that a^nb^n specifically is context-free, as I suggest in the sibling comment, computer scientists who are also AI specialists would recognise a^nb^n immediately, as they would Dyck languages and Reber grammars, because those are standard tests of learnability used to demonstrate various principles, from the good old days of purely symbolic AI, to the brave new world of modern deep learning.

For example, I learned about Reber grammars for the first time when I was trying to understand LSTMs, when they were all the hype in Deep Learning, at the time I was doing my MSc in 2014. Online tutorials on coding LSTMs used Reber grammars as the dataset (because, as with other formal grammars it's easy to generate tons of strings from them and that's awfully convenient for big data approaches).

Btw that's really the difference between a computer scientist and a computer engineer: the scientist knows the theory. That's what they do to you in CS school, they drill that stuff in your head with extreme prejudice; at least the good schools do. I see this with my partner who is 10 times a better engineer than me and yet hasn't got a clue what all this Chomsky hierarhcy stuff is. But then, my partner is not trying to be an AI influencer.

Re: Interview with gwern

#108

Post by [deleted] »

Tenoke wrote: post #58
mola wrote: post #44

Everything I read from gwern has this misanthropic undertones. It's hard to put a finger on it exactly, but it grits me when I try reading him. It is also kinda scary that so many people are attracted to this. It rhymes with how I feel about Ayn Rand. Her individualism always seems so misanthropic, her adherants scare me

Everything? He also writes things like this: "A list of unheralded improvements to ordinary quality-of-life since the 1990s going beyond computers." https://gwern.net/improvement

[deleted]

Re: Interview with gwern

#109

Post by tim333 »

fallingsquirrel wrote: post #100
mynegation wrote: post #85

Earlier quoted context omitted.

That works in reverse too. While I am in awe of what humanity already achieved - when I read fictional timelines of fictional worlds (Middle-Earth or Westeros/Essos) I am wondering how getting frozen in medieval like time is even possible. Like, what are they _doing_?

They're probably doing the same thing humans on our earth were doing for centuries until ~1600. Surviving. Given how cruel nature is I think we're lucky to have the resources to do more than just survive, to build up all this crazy technology we don't strictly need to live, just for fun/profit.

Most people get on with life without inventing much new stuff themselves. It was interesting trekking in Nepal that you could go to places without electricity or cars and life went on really quite similar to before and probably still does. Though they may have got solar electric and phones now - not quite sure of the latest status.

Re: Interview with gwern

#110

Post by tim333 »

robinhouston wrote: post #8

It's been a bumper week for interesting podcast interviews with an AI theme! In addition to this, there are Lex Fridman's series of interviews with various key people from Anthropic [0], and a long discussion between Stephen Wolfram and Eliezer Yudkowsky on the theme of AI risk [1]. 0. https://youtu.be/ugvHCXCOmm4 1. https://youtu.be/xjH2B_sE_RQ

Yeah, and I thought Altman + Tan was good https://youtu.be/xXCBz_8hM9w
Post reply on HN