Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The threat is comfortable drift toward not understanding what you're doing

ergosphere.blog

Post reply on HN
Print view View original

431–440 of 668 posts

Re: The threat is comfortable drift toward not understanding what you're doing

#431

Post by silver_silver »

Wowfunhappy wrote: post #386
atomicnumber3 wrote: post #383

Earlier quoted context omitted.

I think we'll get there. We need to get at least some AI bust going first though. It's impossible to talk sense into people who think AI is about to completely replace engineers, or even those who think that, while it might not replace engineers, it's going to be doing 100% of all coding within a year. Or even that it can do 100% of coding right now. There's a couple unfortunate truths going on all at the same time:…

> Or even that [AI] can do 100% of coding right now. I don't actually think the article refutes this. But the AI needs to be in the hands of someone who can review the code (or astrophysics paper), notice and understand issues, and tell the AI what changes to make. Rinse, repeat. It's still probably faster than writing all the code yourself (but that doesn't mean you can fire all your engineers). The question is, how…

My boss decreed the other day that we’re all to start maximising our use of agents, and then set an accordingly ambitious deadline for the current project. I explained that being relatively early in my career I’ve been hesitant to use any kind of LLM so I can gain experience myself (to say nothing of other concerns), and yeah in his words I’ve “missed the opportunity”

Re: The threat is comfortable drift toward not understanding what you're doing

#432

Post by ChrisMarshallNY »

nothinkjustai wrote: post #419
ChrisMarshallNY wrote: post #315

This is not wrong, but the "Bob and Alice" conundrum is not simple, either. In academia, understanding is vital. The same for research. But in production , results are what matters. Alice would be a better researcher, but Bob would be a better producer. He knows how to wrangle the tools. Each has its value. Many researchers develop marvelous ideas, but struggle to commercialize them, while production-oriented enginee…

If results are what matters, why is popular software so buggy and lacking in features?

Because people will pay for crap.

As long as that’s the case, those that create crap will thrive.

Pretty basic, and long predates LLMs.

Re: The threat is comfortable drift toward not understanding what you're doing

#433

Post by boothby »

Wowfunhappy wrote: post #411
senordevnyc wrote: post #407

Earlier quoted context omitted.

Article is paywalled, so perhaps you could just summarize his proposal?

> At the type of place where I taught until recently—a small, selective, private liberal-arts college—administrators can go quite far in limiting AI use, if they have the guts to do so. They should commit to a ruthless de-teching not just of classrooms but of their entire institution. Get rid of Wi-Fi and return to Ethernet, which would allow schools greater control over where and when students use digital technologi…

> If students want to type notes in class or papers in the library, they can use digital typewriters, which have word processing but nothing else.

Only, replacing the guts of such a machine to contain a local LLM is damn easy today. Right now the battery mass required to power the device would be a giveaway, but inference is getting energetically cheaper.

> Colleges that are especially committed to maintaining this tech-free environment could require students to live on campus, so they can’t use AI tools at home undetected.

Just like my on-campus classmates never smoked weed or drank underage, I'm sure.

Re: The threat is comfortable drift toward not understanding what you're doing

#434

Post by itmitica »

techblueberry wrote: post #239
itmitica wrote: post #223

Contrarian just for the sake of it. Get on board or stay behind. Whatever good or bad AI brings to the table, it's here to stay. The cat's out of the bag. Might as well enjoy it. Evolution will not stay on your whimsical made-up reality. It will run you over.

What if AI in the long run makes us slower and less effective. As someone who is one of the folks supercharged by these tools, I could see it. I think people are underestimating the level of experience and knowledge that’s required to prompt LLM’s. Not in the micro sense but in the macro. It seems so easy because it feels easy. But if you don’t have deep understanding of the domain, it will just feel impossible. The…

My experience is AI helping me unload things I am not built for. It allows me to be creative with less drag. Not everyone aims to be useless human automaton or useless human thesaurus. It allows me to exercise my intelligence freely.

Re: The threat is comfortable drift toward not understanding what you're doing

#435

Post by argee »

steveBK123 wrote: post #211

For the people arguing that the output is the code and the faster we generate it the better.. I do wonder where all the novel products produced by 10x devs who are now 100x with LLMs, the “idea guys” who can now produce products from whole clothe without having to hire pesky engineers.. where is the one-man 10 billion dollar startups, etc? We are 3-4 years into this mania and all I see on the other end of it is the L…

Going 100x faster is the problem. As they say, “slow is smooth, and smooth is fast,” when going for product market fit this is very important (and understated, in my opinion). It doesn’t help that your thread is spinning out at a hundred yards a second when what you’re doing is trying to thread the needle.

Re: The threat is comfortable drift toward not understanding what you're doing

#436

Post by adriand »

conception wrote: post #397
dragontamer wrote: post #354

Earlier quoted context omitted.

In the US it is dying off. Not so in plenty of other countries. Hopefully US reverses the anti-science trend before it's too late

These movements are growing in every western nation. The trend has been growing over decades. It would be nice to see it reverse but seems unlikely before calamity.

It’s a deliberate process powered by rightwing and capitalist interests designed to create a dumber, less educated and more distracted population. A war as stupid as the one with Iran would not have been possible three decades ago. As ill-advised as the Iraq war was, Bush at least spent months explaining the rationale and building support for it, successfully. Now that’s not needed.

I saw interviews with young Americans on spring break and they were so utterly uninformed it was mind-blowing. Their priorities are getting drunk and getting laid while their country bombs a nation “into the stone ages”, according to their president. And it’s not their fault: they are the product of a media environment and education system designed for exactly this outcome.

Re: The threat is comfortable drift toward not understanding what you're doing

#437

Post by lowsong »

sd9 wrote: post #6

The thing is, agents aren’t going away. So if Bob can do things with agents, he can do things. I mourn the loss of working on intellectually stimulating programming problems, but that’s a part of my job that’s fading. I need to decide if the remaining work - understanding requirements, managing teams, what have you - is still enjoyable enough to continue. To be honest, I’m looking at leaving software because the job…

> agents aren’t going away

Why not? Once the true cost of token generation is passed on to the end user and costs go up by 10 or 100 times, and once the honeymoon delusion of "oh wow I can just prompt the AI to write code" fades, there's a big question as to if what's left is worth it. If it isn't, agents will most certainly go away and all of this will be consigned to the "failed hype" bin along with cryptocurrency and "metaverse".

Re: The threat is comfortable drift toward not understanding what you're doing

#438

Post by grey-area »

Henchman21 wrote: post #426
grey-area wrote: post #395

Earlier quoted context omitted.

Why use a tool that generates plausible garbage?

Because there is no appreciable difference between outputs. Most of the work that most of us do isn't important. It's busywork byproducts of making widgets that most people don't even need. So if your job is already pointless why not make it easier using LLMs?

Sounds a little sad. I think I’d rather find another job.

Re: The threat is comfortable drift toward not understanding what you're doing

#439

Post by jayd16 »

conception wrote: post #310
Wowfunhappy wrote: post #227

> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…

Sadly I don’t see how our current social paradigm works for this. There is no history of any sort of long planning like this or long term loyalty (either direction) with employees and employers for this sort of journeyman guild style training. AI execs are basically racing, hoping we won’t need a Schwartz before they are all gone. But what incentives are in place to high a college grad, have them work without llms fo…

Some folks need to touch the hot stove before they learn but eventually they learn.

If AI output remains unreliable then eventually enough companies will be burned and management will reinstate proper oversight. All while continuing to pay themselves on the back.

Re: The threat is comfortable drift toward not understanding what you're doing

#440

Post by grey-area »

therealdrag0 wrote: post #424
grey-area wrote: post #395

Earlier quoted context omitted.

Why use a tool that generates plausible garbage?

Because I’m skilled enough to use a tool that generates plausible garbage to be more productive than those who don’t use it at making non-garbage.

Are you sure you’re more productive?

Doesn’t sound like these tools should be used to write scientific papers for example and they seem to bamboozle people far more than help them.

Post reply on HN