Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

BYU study: Why some people choose not to use artificial intelligence

news.byu.edu

Post reply on HN
Print view View original

31–40 of 48 posts

Re: BYU study: Why some people choose not to use artificial intelligence

#31

Post by andy99 »

BoiledCabbage wrote: post #15
pico303 wrote: post #11

I’m a dev working with AI to build tools for others, but I don’t use them personally. Why? Because they make your writing sound like everyone else, they produce shoddy and broken code (unless you’re doing something really commonplace), and they dull your own creativity. If you’re relying on someone else to do your work, you’re going to lose the ability to think for yourself. AI is built essentially on averages. It’s…

> AI is built essentially on averages. It is, but that also means if you prompt it correctly it will give you the answer of the average graduate student working on theoretical physics, or the average expert on the historical inter-cultural conflict of the country you are researching. Averages can be very powerful as well.

Research has no average, there's opinion and experience and nuance. This whole "graduate level" thing (no idea if that's what the parent comment refers to) is so stupid, and marketing at people who have never done research or advanced studies.

Getting an average response by necessity gives you something dumbed down and smoothed over that nobody in the field would actually write (except maybe to train and LLM or contribute an encyclopedia entry).

Not that having general knowledge is a bad thing, but LLM output is not representative of what a researcher would do or write.

Re: BYU study: Why some people choose not to use artificial intelligence

#32

Post by stackskipton »

standardUser wrote: post #28
ofjcihen wrote: post #24

Earlier quoted context omitted.

In my experience: Yes. Doing security reviews for this content can be a real nightmare. To be fair though I have no issue with using LLM created code with the caveat being YOU MUST BE UNDERSTAND IT. If you don’t understand it enough to be able to review it you’re effectively copying and pasting Stack Overflow

At least with Stack Overflow there's upvotes and comments to give me some confidence (sometimes too much confidence). With LLMs I start hyper-skeptical and remain hyper-skeptical - there's really no way to develop confidence in it because the mistakes can be so random and dissimilar to the errors we're used to parsing in human-generated content. Having said that, LLMs have saved me a ton of time, caught my dumb error…

Also with most Stack Overflow copy and pasted code, you can Google the suspicious code, find the link to it and read over the question/comments and somewhat grok the decision and maybe even find a fix in the comments.

Most AI Code does not have prompts and even if it does, there is not guarantee that same prompt will produce the same output so it's like reading human code except human can't explain themselves even if you have access to them.

Re: BYU study: Why some people choose not to use artificial intelligence

#34

Post by Jach »

karmakaze wrote: post #25
mdaniel wrote: post #13

I wish OP had linked to the actual study, because that blurb is a press release Resistance to Generative AI: Investigating the Drivers of Non-Use - https://scholarspace.manoa.hawaii.edu/server/api/core/bitstr...

Either I'm an outlier or this is a bad article/study. All the reasons given are fears: Output Quality - Fears that... Ethical - Fears about.. Risk - Fears that... Human Connection - Fears that... Impairment - Fears that... Creativity - Fears that... My disuse is all about flow and value, not fear. The ways I use it is in refining ideas at a higher level, not outputting code/content/etc (except for rote work).

Programmers are usually the minority. The introduction mentions that ChatGPT reached 100 million users faster than any other consumer technology in history. There aren't even that many programmers worldwide. In their table 3 of non-use scenarios, programming isn't an explicit one while "creating poetry" is. (Despite mentioning CoPilot use as one of their pre-screen options. Perhaps in the 24 situation codes they came up with, one of the 4 they removed for table 3 due to having the greatest reported AI usage was programming, as this study is more about non-usage.) To put yourself in the mindset of a study participant, go through each of those scenarios and ask yourself if you've used the AI for that (and would use it again) or not, and why.

They also only surveyed a few hundred people via Prolific.

The product success (millions of users) implies that for most people, concerns over "ease of use" (which is what I'd code your reason of "flow" as) aren't common, because it's quite easy to use for many scenarios. But I'd still expect the concern to come up for those talking about using it for artwork because even with things like inpainting in a graphics editor it's still not exactly easy to get exactly what you want... The study mentions they consolidated 29 codes into the 8 in table 2 (you missed the two general concerns, Societal and Barrier). Perhaps "ease of use" slides ito "Barrier", as they highlight "lack of skill" as fitting there and that's similar. It would be nice to see a sample of the actual survey questions and answers and coding decisions but hey what is open data am I right.

Anyway, the table headings are "General Concerns" and "Specific Concerns". I wouldn't get too hung up on the use of the term "fear" as the authors seem to use it interchangeably with "concern". I'd also read something like "Output Quality: fears that GenAI output is inaccurate..." synonymously as "has low confidence in the output quality of GenAI". (I'd code your "value" issue as one about output quality.) All of these fears/concerns/confidence questions can be justifiable, too, the term is independent of that.

Re: BYU study: Why some people choose not to use artificial intelligence

#36

Post by stackskipton »

standardUser wrote: post #21
pico303 wrote: post #11

I’m a dev working with AI to build tools for others, but I don’t use them personally. Why? Because they make your writing sound like everyone else, they produce shoddy and broken code (unless you’re doing something really commonplace), and they dull your own creativity. If you’re relying on someone else to do your work, you’re going to lose the ability to think for yourself. AI is built essentially on averages. It’s…

> they produce shoddy and broken code We must have dramatically different approaches to writing code with LLMs. I would never implement AI-written code that I can't understand or prove works immediately. Are people letting LLMs write entire controllers or modules and then just crossing their fingers?

Yes. VAST majority of developers are working in feature factories where they pick a Jira ticket off the top, it probably has timeboxed work amount. Their goal is close Jira ticket within timeboxed amount by getting build to go green and PM to accept "Yep, that feature was implemented." If badly written LLMs will get build to go green and feature to be accepted, whatever, Jira ticket closed, paycheck collected. Any downstream problems are tomorrow problem when tech debt piles up high enough and Jira tickets to fix the tech debt are written.

Re: BYU study: Why some people choose not to use artificial intelligence

#37

Post by ofjcihen »

Jach wrote: post #30
ofjcihen wrote: post #18

Earlier quoted context omitted.

I see this argument all the time. That the user must not be prompting correctly. In my experience the way you prompt is less important than the “averageness” of the answer you’re looking for.

Talking about averages is really misleading. Talk about capabilities instead, framed in tool language if you must. Quoting https://buttondown.com/hillelwayne/archive/ai-is-a-gamechang... about https://zfhuang99.github.io/github%20copilot/formal%20verifi... "In the post, Cheng Huang claims that Azure successfully used LLMs to examine an existing codebase, derive a TLA+ spec, and find a production bug in that spec." Th…

Take it from someone in the business of exploiting race conditions for money: that’s about as average as you can get. Additionally, whatever Azure is considering “traditional” methods may be bare bones poorly optimized automated code reviews given the egregious issues they’ve had in the past.

As a side note:LLMs by definition do not demonstrate “understanding” of anything.

Re: BYU study: Why some people choose not to use artificial intelligence

#38

Post by ofjcihen »

bawolff wrote: post #14

The weirdest thing about AI is how shocked people that like AI are that some people don't use it. While i'm sure its a useful tool in some situations, and i don't begrudge anyone who finds value in it, it simply doesn't fit into my life as something that would be useful on a regular basis.

Any time I’ve had someone express this kind of thinking they’ve either been a non-dev or someone who writes CRUD exclusively.

Devs and others recognize that the tech use very useful but not “magic” .

Re: BYU study: Why some people choose not to use artificial intelligence

#39

Post by Jach »

andy99 wrote: post #31
BoiledCabbage wrote: post #15

Earlier quoted context omitted.

> AI is built essentially on averages. It is, but that also means if you prompt it correctly it will give you the answer of the average graduate student working on theoretical physics, or the average expert on the historical inter-cultural conflict of the country you are researching. Averages can be very powerful as well.

Research has no average, there's opinion and experience and nuance. This whole "graduate level" thing (no idea if that's what the parent comment refers to) is so stupid, and marketing at people who have never done research or advanced studies. Getting an average response by necessity gives you something dumbed down and smoothed over that nobody in the field would actually write (except maybe to train and LLM or contr…

One thing the "graduate level" concept reminds me of is Terence Tao's semi endorsement almost a year ago: https://mathstodon.xyz/@tao/113132502735585408 People quote the "The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, (static simulation of a) graduate student." part but ignore all the rest of the nuance in the thread like "It may only take one or two further iterations of improved capability (and integration with other tools, such as computer algebra packages and proof assistants) until the level of "(static simulation of a) competent graduate student" is reached, at which point I could see this tool being of significant use in research level tasks." or "I inadvertently gave the incorrect (and potentially harmful) impression that human graduate students could be reductively classified according to a static, one dimensional level of “competence”."

Re: BYU study: Why some people choose not to use artificial intelligence

#40

Post by lucas_membrane »

karmakaze wrote: post #25
mdaniel wrote: post #13

I wish OP had linked to the actual study, because that blurb is a press release Resistance to Generative AI: Investigating the Drivers of Non-Use - https://scholarspace.manoa.hawaii.edu/server/api/core/bitstr...

Either I'm an outlier or this is a bad article/study. All the reasons given are fears: Output Quality - Fears that... Ethical - Fears about.. Risk - Fears that... Human Connection - Fears that... Impairment - Fears that... Creativity - Fears that... My disuse is all about flow and value, not fear. The ways I use it is in refining ideas at a higher level, not outputting code/content/etc (except for rote work).

Fear? I suppose that any negative evaluation can be stereotyped as fear or lack of intrepidity, but perhaps that repeated use of that label is projection -- that the article was written by an AI believer who fears that AI might have to recognize some realities beyond its purview. Or maybe the article was written by AI that has learned that fear is what we fear.

Human thought is implemented by a system that has adapted for hundreds of millions of years in diverse environments. We are adapted to huge variations in resources, threats of innumerable kinds, climates, opportunities, social and ecological relationships, etc, and many of its adaptations may be adaptations to control, balance or modify its other adaptations. It would be crazy to expect human intelligence to be what we could describe as optimized for something, and it would be crazy to expect humans to be able to figure out what that something is even if that were true. Perhaps our minds have gotten us here, and they cannot get us out of here, but they maintain some pretty strong links to our natural environment, which is still our landlord.

AI, OTOH, is a new kind of creature of a single time and a monoculture -- the internet. I don't talk to AI; perhaps someone has asked AI how much fear we should have of AI, and what the odds are.

Post reply on HN