Live data from Hacker News

Interview with gwern

dwarkeshpatel.com

151–160 of 262 posts

Re: Interview with gwern

#151

Interesting that there's no mention of human biodiversity (aka blacks are dumb), as if you spend five minutes on #lesswrong you'll notice that that's a big issue for gwern and the other goons.

Is #lesswrong an irc channel? I’ve never been there, but I can say I’ve never seen that discussion on lesswrong the website

Re: Interview with gwern

#152
post #125

Earlier quoted context omitted.

>> This is a bit stark: there are many great knowledgeable engineers and scientists who would not get your point about a^nb^n. It's impossible to know 100% of of such a wide area as "AI and CS". I think, engineers, yes, especially those who don't have a background in academic CS. But scientists , no, I don't think so. I don't think it's possible to be a computer scientist without knowing the difference between a regu…

Strong gatekeeping vibes. "Not even wrong" is perfect for this sort of fixation with labels and titles and an odd seemingly resentful take that gwern has being an AI influencer as a specific goal.

"not even wrong" is supposed to refer to a specific category of flawed argument, but of course like many other terms it's come to really mean "low status belief"

Re: Interview with gwern

#153
post #146

Earlier quoted context omitted.

Yes, I'm talking about learning neural nets with gradient descent. See also the nice paper I linked below. >> How do you do intelligence without computation though? Beats me! Unlike everyone else in this space, it seems, I haven't got a clue how to do intelligence at all, with or without computation. Edit: re infinite languages, I liked something Walid Saba (RIP) pointed out on Machine Learning Street Talk, that sure…

What I don't know how to do is bounded rationality. Iterating over all the programs weighted by length (with dovetailing if you're a stickler) is "easy", but won't ever get anywhere. And you can't get away with the standard tricky tricks that people use to say it isn't easy, logical induction exists.

Right! See my (long) edit.

Re: Interview with gwern

#154
post #62

Earlier quoted context omitted.

Have you got any real evidence behind your first paragraph?

I don't follow gwern's work closely. But I do know he created an enormous dataset of anime images used to train machine learning and generative AI models [1]. Hosting large datasets is moderately expensive - and it's full of NSFW stuff, so he's probably not having his employer or his college host it. Easy for someone on a six-figure salary, difficult for a person on $12k/year. Also, I thought these lesswrong folks we…

Hosting large datasets can be expensive but the hosting for the danbooru datasets was not. It's "only" a few terabytes in size. A previous release was 3.4TB, so the latest is probably some hundreds of GB, to a TB~, in size larger. The download was hosted on a hetzner IP, which is a provider known for cheap servers. You can pay them $50/m for a server with "unmetered" 1gigabit up/down network + 16TB of disks. $600 a year would not be difficult.

Re: Interview with gwern

#155

Earlier quoted context omitted.

Car and motor vehicles in general get you to work and help you do your work. They don't do the work. I guess that's the difference in thinking. I'm not sure that it's acrually correct: I don't think we'll actually see "AI" actually replace work in general as a concept. Unless it can quite literally do everything and anything, there will always be something that people can do to auction their time and/or health to acq…

Compilers also do the work. As do tractors. That doesn't mean there's no work left to do.

They don't do the work, they help you do the work. The work isn't compiling or ploughing, it's writing software and farming, respectively. Both of which are actually just means to the ends of providing some service that someone will pay "human life tokens" for.

AI maximalists are talking about breaking that pattern and having AI literally do the job and provide the service, cutting out the need for workers entirely. Services being provided entirely autonomously and calories being generated without human input in the two analogies.

I'm not convinced by that at all: if services can be fully automated, who are you going to sell ERP or accounting software to, say? What are people going to use as barter for those calories if their time and bodies are worthless?

But I can see why that is a saleable concept to those who consider the idea of needing to pay workers to be a cruel injustice. Though even if it works at all, which, as I said, I dont believe, the actual follow-on consequences of such a shift are impossible to make any sensible inferences about.

Re: Interview with gwern

#156
post #78

> I love the example of Isaac Newton looking at the rates of progress in Newton's time and going, “Wow, there's something strange here. Stuff is being invented now. We're making progress. How is that possible?” And then coming up with the answer, “Well, progress is possible now because civilization gets destroyed every couple of thousand years, and all we're doing is we're rediscovering the old stuff.” The link in th…

That works in reverse too. While I am in awe of what humanity already achieved - when I read fictional timelines of fictional worlds (Middle-Earth or Westeros/Essos) I am wondering how getting frozen in medieval like time is even possible. Like, what are they _doing_?

Those stories are inspired (somewhat) by the dark ages. Stagnation is kinda the default state of mankind. Look at places like Afghanistan. Other than imported western tech, it's basically a medieval society. Between the fall of the Roman Empire and the middle medieval era, technology didn't progress all that much. Many parts of the world were essentially still peasant societies at the start of the 20th century.

All you really need is a government or society that isn't conducive to technological development, either because they persecute it or because they just don't do anything to protect and encourage it (e.g. no patent system or enforceable trade secrets).

Even today, what we see is that technological progress isn't evenly distributed. Most of it comes out of the USA at the moment, a bit from Europe and China. In the past there's usually been one or two places that were clearly ahead and driving things forward, and it moves around over time.

The other thing that inspires the idea of a permanent medieval society is archaeological narratives about ancient Egypt. If you believe their chronologies (which you may not), then Egyptian society was frozen in time for thousands of years with little or no change in any respect. Not linguistic, not religious, not technological. This is unthinkable today but is what academics would have us believe really happened not so long ago.

Re: Interview with gwern

#157
post #78

> I love the example of Isaac Newton looking at the rates of progress in Newton's time and going, “Wow, there's something strange here. Stuff is being invented now. We're making progress. How is that possible?” And then coming up with the answer, “Well, progress is possible now because civilization gets destroyed every couple of thousand years, and all we're doing is we're rediscovering the old stuff.” The link in th…

That works in reverse too. While I am in awe of what humanity already achieved - when I read fictional timelines of fictional worlds (Middle-Earth or Westeros/Essos) I am wondering how getting frozen in medieval like time is even possible. Like, what are they _doing_?

You're right, really: it's not possible. It's a problem with the conservative impulse (*for a very specific meaning of conservative) in fiction: things don't stay frozen in amber like that. If it was nonfiction - aka real life - the experience of life itself from the perspective of living people would change and transform rapidly in the century view.

Re: Interview with gwern

#158
post #125

Earlier quoted context omitted.

Strong gatekeeping vibes. "Not even wrong" is perfect for this sort of fixation with labels and titles and an odd seemingly resentful take that gwern has being an AI influencer as a specific goal.

OK, I concede that if I try to separate engineers from scientists it sounds like I'm trying to gatekeep. In truth, I'm organising things in my head because I started out thinking of myself as an engineer, because I like to make stuff, and at some point I started thinking of myself as a scientist, malgré moi, because I also like to know how stuff works and why. I multiclassed, you see, so I am trying to understand exa…

I sure hope nobody ever remembers you being confidently wrong about something. But if they do, hopefully that person will have the grace and self-restraint not to broadcast it any time you might make a public appearance, because they're apparently bitter that you still have any credibility.

Re: Interview with gwern

#159
post #138

Earlier quoted context omitted.

> I am wondering how getting frozen in medieval like time is even possible. Like, what are they _doing_? Not discovering sources of cheap energy and other raw inputs. If you look carefully at history, every rapid period of growth was preceded by a discovery or conquest of cheap energy and resources. You need excess to grow towards the next equilibrium.

1400s Age of Explanation? 1200s Mongol Conquest? BC 100s Roman conquests of the Mediterranean? None match that thesis

They all do?

Age of exploration was powered by finding new sources of slaves and materials in the East (india, asia, also eastern europe to an extent)

The mongol conquest itself was capturing vast sources of wealth (easier to take from others than build yourself)

Same with Rome. Each new conquest brought more slaves and natural resources to the empire. It used this to fuel more expansion.

Post reply on HN