Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Will scaling work?

dwarkeshpatel.com

Post reply on HN
Print view View original

181–190 of 289 posts

Re: Will scaling work?

#181

Post by RaftPeople »

visarga wrote: post #69
beepbooptheory wrote: post #57

Earlier quoted context omitted.

While this is certainly true, I think we can't ignore the intense enthusiasm and faith of a large cohort of our peers (or, you know, HN commenters) who believe this to be The Way, and are not necessarily stakeholders in any meaningful sense. Just look at some of the responses even in this thread. It feels like some people just need this, and respond to balanced skepticism as Alyosha does to his brother Ivan. In part,…

There is no magic in the brain. There is no magic in LLMs. There is just new experience we gain by interacting with the environment and society. And there is the trove of past experience encoded in our books. We got smart by collecting experience, in other words, from outside. The magic in the brain was not in the brain, but everywhere else. What is experience? We are in state S, and take action A, and observe feedba…

> There is no magic in the brain.

Consciousness?

Re: Will scaling work?

#182

Post by RaftPeople »

sgt101 wrote: post #123

>Here’s one of the many astounding finds in Microsoft Research’s Sparks of AGI paper. They found that GPT-4 could write the LaTex code to draw a unicorn. a lot of people have tried to replicate this, I have tried. It's very hard to get GPT-4 to draw a unicorn, also asking it to draw an upside down unicorn is even harder.

A person commenting on this topic at a different site mentioned that there is a lot of content on the internet around how to draw (animals?) with LaTex and a different tool (can't remember the name), so it's unclear if GPT is just regurgitating or if it's generalizing.

Re: Will scaling work?

#183

Post by lnxg33k1 »

slibhb wrote: post #49

The best analogy for LLMs (up to and including AGI) is the internet + google search. Imagine explaining the internet/google to someone in 1950. That person might say "Oh my god, everything will change! Instantaneous, cheap communication! The world's information available at light speed! Science will accelerate, productivity will explode!" And yet, 70 years later, things have certainly changed, but we're living in the…

Imagine telling those same people in the 50s that all those changes in productivity would come for the benefit of no one since the work week would be the same and purchasing power would decline

Re: Will scaling work?

#184

Post by cortic »

lossolo wrote: post #120
cortic wrote: post #34

Earlier quoted context omitted.

If humans are basically evolved LLMs, which i think is likely; Reasoning will be an emergent property of LLMs within context with appropriate weights.

Reasoning and intelligence exists without language.

You know i assumed that was true until right now. But I can't think of a single example of reason and intelligence existing without any form of language. Even insects have rudimentary language, and in fact reasoning and intelligence seem to scale with the complexity of language, both by species and within species.

Re: Will scaling work?

#185

Post by bee_rider »

Negitivefrags wrote: post #166
bee_rider wrote: post #60

Earlier quoted context omitted.

The internet did change things pretty dramatically. Productivity at information communication tasks just isn’t the entire economy. I think we are massively more productive. Some of the biggest new companies are ad companies (Google, Facebook), or spend a ton of their time designing devices that can’t be modified by their users (Apple, Microsoft). Even old fashioned companies like tractor and train companies have time…

I remember long ago reading an argument that information technology has not actually increased productivity. I really wish I could find a source for this now, but I just can't seem to find it anywhere on the internet. Here it is anyway: The administration of the Tax Service uses 4% of the total tax revenue it generates. This percentage has stayed relatively fixed over time. If IT really improved productivity, wouldn'…

Hmm, I’m not sure I buy it, because I’m not sure what additional effort applied to tax administration looks like.

Perhaps we could be optimistic about people and assume the amount of real, legitimate tax fraud and evasion is pretty low. If we took the latter scenario you present—increasing efficiency means the same amount of people will do more work—and assumed this effort is instead applied to decreasing the number of random errors (which might result in someone overpaying or underpaying), we wouldn’t necessarily expect to see a change in the expected value of the taxes. But, it could be “better” in the sense that it is more fair.

Re: Will scaling work?

#186

Post by jodrellblank »

herval wrote: post #171
ketzo wrote: post #152

Earlier quoted context omitted.

I mean, very broad strokes, but I can see GP’s point. - people eat plants and animals - people pay money for goods and services - there are countries, sometimes they fight, sometimes they work together - men and women come together to create children, and often raise those children together etc, etc, etc The “bones” of what make up a capital-S Society are pretty much the same. None of these things had to stay the sam…

I mean, has _any_ change in _human history_ impacted those considerably? This argument is like saying we live the same way the cavemen did...

I'm not the original commenter, but moving from nomadic tribes to stable settlements, moving from hunter gathering to agriculture, moving from almost everyone subsistence farming to the introduction of money at all, to most people working unrelated for money and trading money for food[2], moving from multigenerational homes to nuclear families to sending kids to schools and daycares, moving from tribal lands to countries with a national identity of their own which you are supposed to have some kind of loyalty to - over and above the king/warlord you trade protection with.

As well as those, the change from food and goods being scarce to abundant roughly corresponding with the industrial revolution (abundant textiles and clothes) and the early to mid 1900s (factories), labour receding from sunrise to sundown changing to a working week with days off (various, but early 1900s official 5 day week[1] and 8 hour day), changing to the more recent thing where both parents have to work to get enough income while the child is away all day, massively increased free time (particularly household chore automation - electricity, light, central heating, food mixers, washing machines, mostly early to mid 1900s).

Compared to those things, the internet gets you something else to read or watch (instead of TV, newspaper, book, radio) and some other way to talk (instead of letter, telegram, postcard, telephone). Yes the organisation of things happens quicker and information comes from farther away, and can be more up to date, but you spend your time sitting in a chair watching or reading (office, home, school) like you did before, you buy things and have them delivered or go collect them (like you did before), you consult maps and directories and consumer advice and government documents (like you did before), you take and share holiday photos (like before). It's different, but it's not all that different.

[1] https://www.bbc.co.uk/bitesize/articles/zf22kmn (1932 in America)

[2] https://researchbriefings.files.parliament.uk/documents/SN03... - the UK had 1.7M people working in farming in 1851, down to 182k today while the population has roughly 4x'd in the same time.

Re: Will scaling work?

#187

Post by berniedurfee »

joe_the_user wrote: post #167
ctoth wrote: post #98

Earlier quoted context omitted.

> I think there’s a huge assumption here that more LLM will lead to AGI. I'm not sure you realize this, but that is literally what this article was written to explore! I feel like you just autocompleted what you believe about large language models in this thread, rather than engaging with the article. Engagement might look like "I hold the skeptic position because of X, Y, and Z, but I see that the other position has…

I'm not sure you realize this, but that is literally what this article was written to explore! Yeah but it's "exploration" answers all the reasonable objections by just extrapolating vague "smartness" (EDITED [1]). "LLMs seem smart, more data will make 'em smarter..." If apparent intelligence were the only measure of where things are going, we could be certain GPT-5 or whatever would reach AGI. But I don't many peopl…

I’ve also heard it said that “apparent” intelligence is good enough to be called “real” intelligence if it’s indistinguishable from the real thing. That’s where I have a strong feeling that we’re missing the true meaning of intelligence, reasoning and consciousness.

As you said, we may very well be a couple iterations away from a chatbot that is truly indistinguishable from a human, but I still strongly assert that even a perfectly coherent chatbot is nothing more than an automaton and we humans are not automatons.

The fact that a couple replies in this thread made me feel defensive and a bit discouraged with their condescending tone is to me an internal reaction that an LLM or similar system will never have. Maybe an appropriate emotional reaction can be calculated and simulated, but I think the nature of the experience itself is truly beyond our current comprehension.

Maybe I’m grasping at the metaphysical to rationalize my fear that we’re on the cusp of understanding consciousness… and it turns out to be pretty boring and will be included with Microsoft O365 in a couple years.

Re: Will scaling work?

#188

Post by berniedurfee »

jeremyjh wrote: post #140
ctoth wrote: post #98

Earlier quoted context omitted.

> I think there’s a huge assumption here that more LLM will lead to AGI. I'm not sure you realize this, but that is literally what this article was written to explore! I feel like you just autocompleted what you believe about large language models in this thread, rather than engaging with the article. Engagement might look like "I hold the skeptic position because of X, Y, and Z, but I see that the other position has…

I feel like an LLM would do a much better job than GP.

Lol, at least then your comment wouldn’t have bothered me so much!

Re: Will scaling work?

#190

Post by jeremyjh »

joe_the_user wrote: post #167
ctoth wrote: post #98

Earlier quoted context omitted.

> I think there’s a huge assumption here that more LLM will lead to AGI. I'm not sure you realize this, but that is literally what this article was written to explore! I feel like you just autocompleted what you believe about large language models in this thread, rather than engaging with the article. Engagement might look like "I hold the skeptic position because of X, Y, and Z, but I see that the other position has…

I'm not sure you realize this, but that is literally what this article was written to explore! Yeah but it's "exploration" answers all the reasonable objections by just extrapolating vague "smartness" (EDITED [1]). "LLMs seem smart, more data will make 'em smarter..." If apparent intelligence were the only measure of where things are going, we could be certain GPT-5 or whatever would reach AGI. But I don't many peopl…

> My favorite measure is "query intelligence" versus "task accomplishment intelligence".

The article does address this regarding abysmal performance on the GitHub PR benchmark. It’s one of the big “ifs” for sure.

Post reply on HN