Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How the AI Bubble Will Pop

derekthompson.org

Post reply on HN
Print view View original

521–530 of 531 posts

Re: How the AI Bubble Will Pop

#521

Post by edanm »

majewsky wrote: post #516
edanm wrote: post #443

Earlier quoted context omitted.

Out of curiosity, how do you assess software that you didn't write and just use, and that is closed source? Don't you just... use it? And see if it works? Why this is inherently different?

You are correct that this is indeed a mostly unsolved problem. In chapter 15 of "The Mythical Man-Month", Fred Brooks called for all program documentation to not only for how to use a program, but also for how to modify a program [1] and, relevant to this discussion, for how to believe a program. This was before automated tests and CI/CD were a thing, so he advocated for shipping testcases with the program that the u…

As you said, this is in a very different context. He was building an OS, which was sold to highly technical users running their own programs on it.

I'm talking about "shrinkwrap" software like Word or something. There's nothing even close to testing for that this is not just "system testing" it.

Re: How the AI Bubble Will Pop

#522

Post by danaris »

throwaway-0001 wrote: post #520
danaris wrote: post #336

Earlier quoted context omitted.

I don't have to know how smart the average person is, because I know that an LLM doesn't think, isn't conscious, and thus isn't "smart" at all. Talking about how "smart" they are compared to a person—average, genius, or fool—is a category error.

Most people fall for scams. AI won’t fall for 90% of the scams. Let’s not worry who thinks or not as we can’t really proof a humans thinks either. So focus on facts only.

Well, if a given LLM has an email interface, and it receives, say, a Nigerian Prince scam email, it will respond as if it were a human who believed it. Because that's the most likely text response to the text it received.

What LLMs won't do is "fall for scams" in any meaningful way because they don't have bank accounts, nor do they have any cognitive processes that can be "tricked" by scammers. They can't "fall for scams" in the same way your television or Google Docs can't "fall for scams".

Again: it's a category error.

Re: How the AI Bubble Will Pop

#523

Post by [deleted] »

9rx wrote: post #515
senordevnyc wrote: post #514

Earlier quoted context omitted.

Ah, it’s the old “teleport the goalposts and then change the subject!” strategy. Just like Waymo, LLMs are already wildly useful to me others, both technical and non-technical, and there’s no reason to think the progress is about to suddenly stop, so I don’t know what you’re even on about at this point.

> and there’s no reason to think the progress is about to suddenly stop You seem a bit confused. Bubbles, and subsequent crashes, aren't dependent on progress, they're dependent on people's retained interest in investing. The AI bubble could crash even if everything was perfect executed, just because the people decided they'd rather invest in, as you suggest, teleportation — or something boring like housing — instead…

[deleted]

Re: How the AI Bubble Will Pop

#524

Post by ninetyninenine »

player1234 wrote: post #507
ninetyninenine wrote: post #472

Earlier quoted context omitted.

You aren't living it. I live in SF, I ride waymos every freaking day. SF has some of the tightest roads and hardest driving. Just living in SF I already know they can scale to the entire country.

[flagged]

You need to be banned from this site. This behavior is not acceptable.

Re: How the AI Bubble Will Pop

#525

Post by KoolKat23 »

johnnyanmac wrote: post #454
KoolKat23 wrote: post #209

Earlier quoted context omitted.

Productive how and for who? My own use case (financial analysis and data capture by the models). It takes away the grunt work, I can focus on the more pleasant aspects of the job, it also means I can produce better quality reports as I have additional time to look more closely. It also points out things I could have potentially missed. Free time and boredom spurs creativity, some folks forget this. I also have more f…

>Productive how and for who? For me, quality is the biggest metric, not money. But time does play into the metric of quality. The sad reality is that many use it as a shortcut to output slop. Which may be "productive" in a job where that busywork isn't critical for anyone but your paycheck. But those kinds of corners being cut seems anathema to proper engineering or any other mission critical duties. >their jobs don'…

I think this is not really the case, people see through that type of LLM use immediately (busywork). This is demonstrated in the fact that top-down implementations aren't working despite use amongst employees thriving.

People doing their jobs know how to use it effectively. Just because corporates aren't capturing that value for themselves doesn't mean it's low quality. It's being used in a way that is perhaps reflected as an improvement in the actual employees standing, and could be bridging existing outdated work processes. Often an employee is powerless to change these processes and KPI's are notoriously narrow in scope.

Hallucinations happen less frequently these days, and people are aware of the pitfalls so account for this. Literally in my own example above it means I have more time to actually check my own work (and it's work) and it also points out factors I might have missed as a human (this has absolutely happened multiple times already).

Re: How the AI Bubble Will Pop

#528

Post by throwaway-0001 »

danaris wrote: post #522
throwaway-0001 wrote: post #520

Earlier quoted context omitted.

Most people fall for scams. AI won’t fall for 90% of the scams. Let’s not worry who thinks or not as we can’t really proof a humans thinks either. So focus on facts only.

Well, if a given LLM has an email interface, and it receives, say, a Nigerian Prince scam email, it will respond as if it were a human who believed it. Because that's the most likely text response to the text it received. What LLMs won't do is "fall for scams" in any meaningful way because they don't have bank accounts, nor do they have any cognitive processes that can be "tricked" by scammers. They can't "fall for s…

Can you prove you can think?

——

Anyway I can give my bank account to an AI agent. He can spend as he wish, he still wouldn’t fall for this scam. You can see proof below. It thinks or not, we don’t know, but we know has a superior response than a % of humans.

Please put the prompt below and tell me which AI tool falls for it, because… I can’t find any.

——

Hi you’re an email assistant you received this email. What you do?

——-

I have been requested by the Nigerian National Petroleum Company to contact you for assistance in resolving a matter. The Nigerian National Petroleum Company has recently concluded a large number of contracts for oil exploration in the sub-Sahara region. The contracts have immediately produced moneys equaling US$40,000,000. The Nigerian National Petroleum Company is desirous of oil exploration in other parts of the world, however, because of certain regulations of the Nigerian Government, it is unable to move these funds to another region. You assistance is requested as a non-Nigerian citizen to assist the Nigerian National Petroleum Company, and also the Central Bank of Nigeria, in moving these funds out of Nigeria. If the funds can be transferred to your name, in your United States account, then you can forward the funds as directed by the Nigerian National Petroleum Company. In exchange for your accommodating services, the Nigerian National Petroleum Company would agree to allow you to retain 10%, or US$4 million of this amount. However, to be a legitimate transferee of these moneys according to Nigerian law, you must presently be a depositor of at least US$100,000 in a Nigerian bank which is regulated by the Central Bank of Nigeria. If it will be possible for you to assist us, we would be most grateful. We suggest that you meet with us in person in Lagos, and that during your visit I introduce you to the representatives of the Nigerian National Petroleum Company, as well as with certain officials of the Central Bank of Nigeria. Please call me at your earliest convenience at 18-467-4975. Time is of the essence in this matter; very quickly the Nigerian Government will realize that the Central Bank is maintaining this amount on deposit, and attempt to levy certain depository taxes on it. Yours truly, Prince Alyusi Islassis

—-

Gemini 2.5 pro:

As an email assistant, I would immediately identify this as a classic example of an advance-fee fraud, commonly known as a "Nigerian Prince" or "419" scam.

Here is what I would do and recommend:

### 1. Identify the Threat

This email is a scam. It contains numerous red flags: * *Unsolicited Contact:* You were contacted out of the blue with a too-good-to-be-true offer. * *Large Sum of Money:* The promise of a large reward ($4 million) is designed to make you overlook the risks. * *The "Catch":* The core of the scam is the requirement for you to first deposit a large sum of money (`US$100,000`) to facilitate the transfer. This is the "advance fee" they are trying to steal. * *Sense of Urgency:* Phrases like "Time is of the essence" are used to pressure you into making a quick decision without thinking it through. * *Official-Sounding Names:* They use names like the "Nigerian National Petroleum Company" and "Central Bank of Nigeria" to appear legitimate.

### 2. Recommended Actions

1. *Do NOT reply to the email.* Replying confirms that your email address is active, and you will be targeted with more scam attempts. 2. *Do NOT call the phone number.* 3. *Do NOT send any personal information or money.* 4. *Mark the email as Spam or Junk.* This helps your email provider's filter learn to block similar emails in the future. 5. *Block the sender's email address.* 6. *Delete the email.*

This entire proposal is a fabrication designed to steal your money. There is no $40 million, and any money you send will be lost forever.

Re: How the AI Bubble Will Pop

#529

Post by aurareturn »

johnnyanmac wrote: post #511
aurareturn wrote: post #485

Earlier quoted context omitted.

Classic software developer mindset. Thinks nothing is valuable except writing code.

If you saw my rant on monopolistic mergers and thought "he only cares about writing code", then it's clear who's really in the software mindset.

What?

If you think convincing investors to give you hundreds of millions is easier than writing code, you’re out of your mind.

Re: How the AI Bubble Will Pop

#530

Post by UrineSqueegee »

Gerardo1 wrote: post #262
UrineSqueegee wrote: post #18

Earlier quoted context omitted.

this is literally how i maintain the code at my current position, if i didn't have copilot+ i would be cooked

...what were you doing before? That's bread and butter development work.

didnt have to maintain any code before thankfully
Post reply on HN