AI assistants: The convenient crutch for new developers or a recipe for disaster

FUD. I used ChatGPT yesterday for some bash scripting. I find ChatGPT useful in situations where "I know what I don't know" e.g., I can never quite remember the array syntax in bash, I forget what flags I want for grep or curl to do the task at hand. Sure, I can read the man page (and I am a big fan of reading the man page), but the AI assistant gets me to my destination much faster after I interactively iterate to arrive at what I want. Yes, I have learned the hard way that you should not trust ChatGPT too much and you should view its output with a lot of skepticism. But overall it is a huge advancement in developer productivity, and can serve as a learning tool as well.

Sure, but I think the imagined hellscape is one where the entire stack of developers don't know what they're doing.

It's like the ostensible toaster company that has outsourced design, engineering, manufacturing, marketing, sales, HR, everything I’ll the way to the janitor. The question then is: what does the toaster company even know about making toasters? If you play this game long enough, then you arrive at a point where entire sectors operate this way. Keep going and the entire economy works this way. At some point we have to stop outsourcing, or we'll lose a lot of knowledge when the original creators of that knowledge kick the bucket. At this point we can't rely on that knowledge being encoded in language models.

I don't think so...

The general populace will use the crippled AIs presented by mainstream corporations and governments.

Anyone who cares about their information will use un-crippled AIs operating without guardrails, which are always available to monitor what they read and hear, and comment on it.

Sort of like when my relatives call me to analyze a potential Phishing email -- I don't do anything difficult; view the raw mail; check the DKIM signing and SPF validity, look at the full URLs of originating servers vs. the claimed From: and the document contents' claimed source, verify all transited MTAs, check the contents for malicious looking payloads, etc.

Any half-decently trained LLM can do all of this, instantly and trivially, and lay out the evidence in an understandable fashion for a non-technical user.

Imagine that, but amplified by an LLM trained to recognize and categorize arguments in a variety of technical fields. Not to tell you what to believe -- but to categorize the claims by its heritage vs. the main groupings of theories in that field, and then give a set of links to highly rated "for" and "against" arguments.

I don't need some LLM trained by a partisan, with guard-rails set up by some corporation's legal and HR departments. I don't think most people will stand for that, when we have laptops that can already run reasonably capable LLMs w/ training on focussed datasets!

> There's no data to back up their alarmist, knee-jerk hand wringing, just like when people thought novels were bad for people [1], or when writing would ruin our memories [2].

Before we knew about germs the "data" suggested that we could bleed out evil tumors to cure illness, or similarly that the sun revolved around the earth. Would you also have labeled such conjecture as alarmist, knee-jerk hand wringing? A lack of data by no means invalidates a hypothesis, on the contrary it encourages further investigation, which leads to scientific progress.

I found it ironic that you referenced Socrates as an example of such alarmist claims, because the way I interpreted what he said is that without truly understanding a thing, and instead relying upon rote recitation, one could only continue to repeat the conclusions that others had painstakingly worked toward, but they would not be able to explain why or how, only that it is.

You need look no further than the embarrassing circus that was "fact checking" during recent political cycles to see that the concerns Socrates held have in fact come to fruition. Without any evidence except for a tweet hordes of people would confidently and voraciously defend a stance they knew nothing about, often times being completely wrong.

SOCRATES: "The same is true of written words. You’d think they were speaking as if they had some understanding, but if you question anything that has been said because you want to learn more, it continues to signify just that very same thing forever. When it has once been written down, every discourse roams about everywhere, reaching indiscriminately those with understanding no less than those who have no business with it, and it doesn’t know to whom it should speak and to whom it should not."

I hope that is the case, but I am doubtful of the overall success. My reasoning being looking at the lack of success of decentralized competition of big tech for search and social media. Other than crypto currency, these other web3 attempts just haven't been largely successful.

Mostly due to the fact the most people will opt for features and convenience over privacy and liberty. The people using Brave, Minds etc will likely use non gated AI, but I suspect the most everyone else will not. Hope I'm wrong.