Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The threat is comfortable drift toward not understanding what you're doing

ergosphere.blog

Post reply on HN
Print view View original

391–400 of 668 posts

Re: The threat is comfortable drift toward not understanding what you're doing

#391

Post by Planktonne »

thepasch wrote: post #381
atoav wrote: post #171

Earlier quoted context omitted.

The thing is Bob can use HammerAsAService™ to put in a nail. It is so cheap! Way cheaper than buying an actual hammer. The problem with unlearning generic tools and relying on ones you rent by big corporations is that it is unreliable in the long term. The prices will be rising. The conditions will worsen. Oh nice that Bob made a thing using HammerAsAService™, but the terms of conditions (changing once a week) he acc…

The majority of nails people might want to rent a HammerAsAService for these days can already easily be put in by open source hammers you can run on consumer, uh… workbenches.

Not to stretch the metaphor too far, but those workbenches require understanding (and hammers) to set up.

Will the paid tools always tell their users how to use the free versions, and if not, how will the users learn to do it independently?

Re: The threat is comfortable drift toward not understanding what you're doing

#393

Post by nothrabannosir »

NiloCK wrote: post #324
nothrabannosir wrote: post #314

Earlier quoted context omitted.

What do people mean exactly when they bring up “Socrates saying things about writing”? Phaedrus? > “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; [275a] and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess. > "Fo…

But did this grind humanity to a halt? Yes - specific faculties atrophied - I wouldn't dispute it. But the (most) relevant faculties for human flourishing change as a function of our tools and institutions.

Someone brought up Socrates upthread:

> People would have said the same about graphing calculators or calculators before that. Socrates said the same thing about the written word.

If the conclusion now becomes “actually, Socrates was correct but it wasn’t that bad”, then why bring up Socrates in the first place?

Re: The threat is comfortable drift toward not understanding what you're doing

#395

Post by grey-area »

Wowfunhappy wrote: post #227

> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…

Why use a tool that generates plausible garbage?

Re: The threat is comfortable drift toward not understanding what you're doing

#396

Post by SoftTalker »

mattmanser wrote: post #62
roenxi wrote: post #40

Earlier quoted context omitted.

That doesn't sound like much of an issue. Bob was already going to encounter problems that are too large and complex for him to solve, agents or otherwise. Life throws us hard problems. I don't recall if we even assumed Bob was unusually capable, he might be one of life's flounderers. I'd give good odds that if he got through a program with the help of agents he'll get through life achieving at least a normal level o…

The authors point went a little over your head. It doesn't matter if Bob can be normal. There was no point to him being paid to be on the program. From the article: If you hand that process to a machine, you haven't accelerated science. You've removed the only part of it that anyone actually needed.

And then you realize that most of science is unnecessary. As TFA points out, it doesn't matter if the age of the universe is 13.77 or 13.79 billion years. So you ban AI in science, you produce more scientists who can solve problems that don't matter. So what?

Re: The threat is comfortable drift toward not understanding what you're doing

#397

Post by conception »

dragontamer wrote: post #354
conception wrote: post #350

Earlier quoted context omitted.

Well, we are already rolling over and dying (literally) on everything from vaccine denial to climate change. So, yes, we are. Obviously yes.

In the US it is dying off. Not so in plenty of other countries. Hopefully US reverses the anti-science trend before it's too late

These movements are growing in every western nation. The trend has been growing over decades. It would be nice to see it reverse but seems unlikely before calamity.

Re: The threat is comfortable drift toward not understanding what you're doing

#398

Post by 8note »

iterateoften wrote: post #359
wongarsu wrote: post #234

Earlier quoted context omitted.

Duh, the only way to get startup experience is indeed to get startup experience. My point is that getting into the weeds of writing CRUD software is not the only way to gain the ability to write complex algorithms, or to debug complex issues, or do performance optimization. It's only common because the stuff you make on the journey used to be economically valuable

> write complex algorithms, or to debug complex issues, or do performance optimization That’s the stuff that ai is eating. The stuff I’m talking about (scaling orgs, maintaining a project long term, deciding what features to build or not build etc) is stuff very hard for ai

I dont know if id call it "hard for ai" so much as "untreaded ground"

agents might be better at it than people are, given the right structure

Re: The threat is comfortable drift toward not understanding what you're doing

#399

Post by mrob »

Wowfunhappy wrote: post #329
conception wrote: post #310

Earlier quoted context omitted.

Sadly I don’t see how our current social paradigm works for this. There is no history of any sort of long planning like this or long term loyalty (either direction) with employees and employers for this sort of journeyman guild style training. AI execs are basically racing, hoping we won’t need a Schwartz before they are all gone. But what incentives are in place to high a college grad, have them work without llms fo…

Then the social paradigm needs to change. Is everyone just going to roll over and die while AI destroys academia (and possibly a lot more)? Last September, Tyler Austin Harper published a piece for The Atlantic on how he thinks colleges should respond to AI. What he proposes is radical —but, if you've concluded that AI really is going to destroy everything these institutions stand for, I think you have to at least co…

>What he proposes is radical

It sounds entirely reasonable and moderate to me.

Re: The threat is comfortable drift toward not understanding what you're doing

#400

Post by abustamam »

didgetmaster wrote: post #253
NiloCK wrote: post #162

Earlier quoted context omitted.

People would have said the same about graphing calculators or calculators before that. Socrates said the same thing about the written word. The determining factor is always "did I come up with this tool". Somehow, subsequent generations always manage to find their own competencies (which, to be fair, may be different). This isn't guaranteed to play out, but it should be the default expectation until we actually see g…

If you hand a broken calculator to someone who knows how to do math, and they entered 123 + 765 which produced an answer of 6789; they should instantly know something is wrong. Hand that calculator to someone who never understood what the tool actually did but just accepted whatever answer appeared; and they would likely think the answer was totally reasonable. Catching an LLM hallucinating often takes a basic unders…

One time when I was a kid I was playing with my older sister's graphing calculator. I had accidentally pressed the base button and now was in hex mode. I did some benign calculation like 10+10 and got 14. I believed it!

I went to school the next day and told my teacher that the calculator says that 10+10 is 14, so why does she say it's 20?

So she showed me on her calculator. She pressed the hex button and explained why it was 14.

I think a major problem with people's usage of LLMs is that they stop at 10+10=14. They don't question it or ask someone (even the LLM) to explain the answer.

Post reply on HN