Earlier quoted context omitted.
I think this is probably true for most skilled professions. AI is best used in the hands of folks already knowledgeable in the skills/professions they are using it for. I liken it to me googling things as a sysadmin vs. Jane from accounting doing it. The non-tech end user is far more likely to make the problem worse, or install something sketchy from the ad riddled results than I am, or one of my help desk employees…
> sysadmin Another domain where LLMs are very effective at confidently leading people down a messy path. I have a roommate using LLMs to guide him through setting up some ollama stuff in my WSL (I happen to have the half-decent GPU here) and after multiple rounds of the bot trying to get him to do things that were redundant if not in the wrong direction entirely (and vaguely insulting as a matter of course), I had to…
Re: AI outperforms law professors in Stanford Law study
#381Yeah, the models are pretty unreliable for sysadmin type tasks. It also really highlights where the training data was lacking. Trying to use LLMs for PowerShell is an effort in frustration. Even opus constantly hallucinates cmdlets often enough to be borderline useless. Not sure if there was a lack of PowerShell in the training data, or its the Verb-Noun naming convention of the cmdlets that throw it off.