Live data from Hacker News

"ChatGPT said this" Is Lazy

terriblesoftware.org

131–140 of 161 posts

Re: "ChatGPT said this" Is Lazy

#131
post #128

Earlier quoted context omitted.

> someone else will reply with “I asked ChatGPT and it says…” or “According to AI…” I had a consultant I’m working with have an employee do that to me. I immediately insisted that every hour they’ve billed on that person’s name be refunded.

How did that go?

> How did that go?

They removed the analyst from our project and credited the hours along with a bonus.

Re: "ChatGPT said this" Is Lazy

#132

I’ve noticed this trend in comments across the internet. Someone will ask or say something, the someone else will reply with “I asked ChatGPT and it says…” or “According to AI…” ChatGPT is free and available to everyone, and so are a dozen other LLMs. If the person making the comment wanted to know what ChatGPT had to say, they could just ask it themselves. I guess people feel like they’re being helpful, but I just d…

I have coworkers who, on a zoom meeting, will respond to questions and discussions by spamming the zoom chat with paste-dumps of ChatGPT, etc. So frustrating and tiresome.

If possible I just go silent when people start copy and pasting ChatGPT at me. Only works in certain cases like Teams/Slack DMs, but it does remove a distraction from my day.

I start responding again when they can be bothered to formulate their own thoughts.

Re: "ChatGPT said this" Is Lazy

#133
post #23

I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I maintain the neural capacity for forming my own original thoughts and ideas no AI has seen before. If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity. Once someone outsources their brain they are unlikely to keep learning or evol…

I’m in the same boat, and what tipped me there is the ethical non-starter that OpenAI and Anthropic represent. They strip-mined the Web, ripped off copyrighted works in neat space, admitting that going through the proper channels was a waste of business resources. They believe that the entirety of human ingenuity should be theirs at no cost, and then they have the audacity to SELL their ill-gotten collation of that k…

On top of which, the most popular systems are proprietary applications running on someone else's machines. After everything GNU showed us for 40 years, I'm surprised programmers are so quick to hand off so much of their process to non-free SaaSS.

Re: "ChatGPT said this" Is Lazy

#134
post #90
post #69

Earlier quoted context omitted.

It hinges assuming that ChatGPT does not thinking, which is clearly false. Hell, Feynman said as much in 1985. https://www.youtube.com/watch?v=ipRvjS7q1DI

I really don't think Feynman said ChatGPT was "thinking" in 1985.

He was referring to computers and program execution in general.

Re: "ChatGPT said this" Is Lazy

#135
post #129
post #23

I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I maintain the neural capacity for forming my own original thoughts and ideas no AI has seen before. If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity. Once someone outsources their brain they are unlikely to keep learning or evol…

> I do not use AI for engineering work and never will > Once someone outsources their brain they are unlikely to keep learning or evolving from that point It doesn't piss me off, it makes me feel sorry for you. Sorry that you're incapable of imagining those with curiosity who use AI to do exactly what you're claiming they don't - learning. The uncurious were uncurious before AI and remain so. They never asked why, th…

The problem with current LLMs is that they are so sycophantic that they don't tell you when you're asking the wrong questions. I've been down the path of researching something with an LLM, and I came to a conclusion. I later revisited the subject and this time I instead read a longer-form authoritative source and it was clear that "interrogating" the matter meant I missed important information.

Re: "ChatGPT said this" Is Lazy

#136
post #90

Earlier quoted context omitted.

I really don't think Feynman said ChatGPT was "thinking" in 1985.

He was referring to computers and program execution in general.

He was wrong. He wasn't right about everything just because he's reddit's favourite physicist (after Neil Degrass Tyson).

Re: "ChatGPT said this" Is Lazy

#137
post #23

I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I maintain the neural capacity for forming my own original thoughts and ideas no AI has seen before. If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity. Once someone outsources their brain they are unlikely to keep learning or evol…

>I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I [...]

This is a framing issue that's keeping you from doing clearer thinking, faster. LLMs, especially in agentic configurations, are a cognitive tool. Just because many people use a tool unskillfully doesn't mean that tool is useless.

>If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity.

Hmmm...

> Once someone outsources their brain they are unlikely to keep learning or evolving

Agreed. But there are many ways in which people can "outsource their brains". LLMs are one way. Dogmas are another.

>If this pisses you off, ask yourself why.

It would definitely piss me off to see an engineering leader fail to grasp the obvious due to some poorly reasoned beliefs.

Re: "ChatGPT said this" Is Lazy

#138
post #129
post #23

I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I maintain the neural capacity for forming my own original thoughts and ideas no AI has seen before. If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity. Once someone outsources their brain they are unlikely to keep learning or evol…

> I do not use AI for engineering work and never will > Once someone outsources their brain they are unlikely to keep learning or evolving from that point It doesn't piss me off, it makes me feel sorry for you. Sorry that you're incapable of imagining those with curiosity who use AI to do exactly what you're claiming they don't - learning. The uncurious were uncurious before AI and remain so. They never asked why, th…

> User: Question. LLM: Answer. User: Why A? Why B? LLM: Answer. User: But C. What's the tradeoff? LLM: Answer. User: Couldn't also X? LLM: Answer. User: I'm not familiar with Y, explain.

In practice it’s more like:

User: Question. LLM: Answer. User: Why A? LLM: Actually, you’re right, it’s not A, it’s B. User: Why B? LLM: Actually, you’re right, it’s not B, it’s C. User: Why C? LLM: Actually, you’re right, it’s not C, it’s D.

I’ve had this happen pretty much every time I asked an LLM a non-trivial question.

Re: "ChatGPT said this" Is Lazy

#139
post #23

I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I maintain the neural capacity for forming my own original thoughts and ideas no AI has seen before. If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity. Once someone outsources their brain they are unlikely to keep learning or evol…

It is increasingly incredibly important here to make a distinction between using an LLM with and without search. Without search, I agree with you. But e.g. ChatGPT with search enabled is often an invaluable research tool, and dramatically speeds up finding relevant sources. It basically automates the spidering of references and links, and also handles the basic checks for semantic relevance quite well, and this task…

When someone is able to demonstrate they are more efficient than me in my areas of research and expertise by utilizing an LLM, I will at least pay attention to their workflow and investigate ways to replicate it with FOSS tools I can host, improve, debug, and control for myself.

Until then, LLMs are of no interest to me. I am pretty good at using grep on local data caches.

Re: "ChatGPT said this" Is Lazy

#140
post #129
post #23

I do not use AI for engineering work and never will, because doing the work of thinking for myself is how I maintain the neural capacity for forming my own original thoughts and ideas no AI has seen before. If anyone gives me an opinion from an AI, they disrespect me and themselves to a point they are dead to me in an engineering capacity. Once someone outsources their brain they are unlikely to keep learning or evol…

> I do not use AI for engineering work and never will > Once someone outsources their brain they are unlikely to keep learning or evolving from that point It doesn't piss me off, it makes me feel sorry for you. Sorry that you're incapable of imagining those with curiosity who use AI to do exactly what you're claiming they don't - learning. The uncurious were uncurious before AI and remain so. They never asked why, th…

A JR using an (open source, open weight, and locally hosted) LLM to learn a new well established well understood industry skill, I could tolerate if I can see they are clearly improving in ways I can easily validate when pairing and interacting with them when the LLMs are not available.

That said, for me personally, almost all of the work I do is on things that have never been done before in security engineering and supply chain security and the entire body of relevant public research an LLM could have trained on is like 10 people, most of whom I am in frequent touch with and very familiar with their work.

LLMs in general are very very bad at threat modeling or security engineering because there is so little training data on the -right- way to do things. They can often produce code that -works- but most would be unable to spot when it is wildly insecure.

There are many many cases where the overwhelming majority of the way things are done in open source training data are completely wrong, and are what an LLM will respond with on average.

Honestly the only way I could maybe see using an LLM in teaching the type of security engineering and auditing work I do is having LLMs generate code examples, and train humans to spot the security flaws the LLM confidently overlooked because it cannot reason or threat model.

Post reply on HN