Earlier quoted context omitted.
Why use a tool that generates plausible garbage?
Because I’m skilled enough to use a tool that generates plausible garbage to be more productive than those who don’t use it at making non-garbage.
[deleted]
The threat is comfortable drift toward not understanding what you're doing
451–460 of 668 posts
Re: The threat is comfortable drift toward not understanding what you're doing
#452Earlier quoted context omitted.
Because I’m skilled enough to use a tool that generates plausible garbage to be more productive than those who don’t use it at making non-garbage.
Are you sure you’re more productive? Doesn’t sound like these tools should be used to write scientific papers for example and they seem to bamboozle people far more than help them.
Yes I’m sure I’m more productive. I have decades of experience before AI to compare to.
I can only speak to the engineering context not academia, but I would expect there’s similar patterns of busywork. Even the blog authors admit to using AI. AI cant replace thinking but it can replace menial labor, which is prevalent even among “knowledge workers”.
Re: The threat is comfortable drift toward not understanding what you're doing
#453Earlier quoted context omitted.
Sadly I don’t see how our current social paradigm works for this. There is no history of any sort of long planning like this or long term loyalty (either direction) with employees and employers for this sort of journeyman guild style training. AI execs are basically racing, hoping we won’t need a Schwartz before they are all gone. But what incentives are in place to high a college grad, have them work without llms fo…
Then the social paradigm needs to change. Is everyone just going to roll over and die while AI destroys academia (and possibly a lot more)? Last September, Tyler Austin Harper published a piece for The Atlantic on how he thinks colleges should respond to AI. What he proposes is radical —but, if you've concluded that AI really is going to destroy everything these institutions stand for, I think you have to at least co…
I was pretty interested until I got to this part:
> Another reason that a no-exceptions policy is important: If students with disabilities are permitted to use laptops and AI, a significant percentage of other students will most likely find a way to get the same allowances, rendering the ban useless. I witnessed this time and again when I was a professor—students without disabilities finding ways to use disability accommodations for their own benefit. Professors I know who are still in the classroom have told me that this remains a serious problem.
This would be a huge problem for students with severe and uncorrectable visual impairments. People with degenerative eye diseases already have to relearn how to do every single thing in their life over and over and over. What works for them today will inevitably fail, and they have to start over.
But physical impairments like this are also difficult to fake and easy to discern accurately. It's already the case that disability services at many universities only grants you accommodations that have something to do with your actual condition.
There are also some things that are just difficult to accommodate without technology. For instance, my sister physically cannot read paper. Paper is not capable of contrast ratios that work for her. The only things she can even sometimes read are OLED screens in dark mode, with absolutely black backgrounds; she requires an extremely high contrast ratio. She doesn't know braille (which most blind people don't, these days) because she was not blind as a little girl.
Committed cheaters will be able to cheat anyway; contemporary AI is great at OCR. You'll successfully punish honest disabled people with a policy like this but you won't stop serious cheaters.
Re: The threat is comfortable drift toward not understanding what you're doing
#454[dead]
Re: The threat is comfortable drift toward not understanding what you're doing
#455> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…
> Which means we need people like Alice! We have to make space for people like Alice, and find a way to promote her over Bob
The solution is relatively simple though - not sure the article suggests this as I only skimmed through:
Being good in your field doesn't only mean pushing articles but also being able to talk about them. I think academia should drift away from written form toward more spoken form, i.e. conferences.
What if, say, you can only publish something after presenting your work in person, answer questions, etc? The audience can be big or small, doesn't matter.
It would make publishing anything at all more expensive but maybe that's exactly what academia needs even irrespective of this AI craze?
Re: The threat is comfortable drift toward not understanding what you're doing
#456> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…
>And so the paradox is, the LLMs are only useful† if you're Schwartz
For so many workers, their companies just want them to produce bullshit. Their managers wouldn't frame it this way, but if their subordinates start producing work with strict intellectual rigor it's going to be an issue and the subordinates will hear about it.
So, you're not wrong. But the majority of LLM customers don't care and they just want to report success internally, and the product needs to be "just good enough." An LLM might produce a shitty webpage. So long as the page loads no on will ever notice or care that it's wrong in the way that a physics paper could be wrong.
Re: The threat is comfortable drift toward not understanding what you're doing
#457> Schwartz's experiment is the most revealing, and not for the reason he thinks. What he demonstrated is that Claude can, with detailed supervision, produce a technically rigorous physics paper. What he actually demonstrated, if you read carefully, is that the supervision is the physics. Claude produced a complete first draft in three days. It looked professional. The equations seemed right. The plots matched expecta…
> Which means we need people like Alice! We have to make space for people like Alice, and find a way to promote her over Bob The solution is relatively simple though - not sure the article suggests this as I only skimmed through: Being good in your field doesn't only mean pushing articles but also being able to talk about them. I think academia should drift away from written form toward more spoken form, i.e. confere…
Essentially a PhD thesis style grilling to replace the current text slop
Re: The threat is comfortable drift toward not understanding what you're doing
#458I was in academia in the pre-GPT-3 era and I don't see a difference between the superficial pass-the-criteria understanding of things then and now. People already rely on a ton of sources, putting their faith into it, recent replication crisis in social sciences had nothing to do with any LLMs.
The problem of academia lies in the first paragraph of this article - supervisor that has to choose doing incremental, clearly feasible stuff. Currently it's called science, but I like to call it knowledge engineering because you're pretty much following a recipe and there is a clear bound on returns to such activities.
Re: The threat is comfortable drift toward not understanding what you're doing
#459Earlier quoted context omitted.
> Or even that [AI] can do 100% of coding right now. I don't actually think the article refutes this. But the AI needs to be in the hands of someone who can review the code (or astrophysics paper), notice and understand issues, and tell the AI what changes to make. Rinse, repeat. It's still probably faster than writing all the code yourself (but that doesn't mean you can fire all your engineers). The question is, how…
My boss decreed the other day that we’re all to start maximising our use of agents, and then set an accordingly ambitious deadline for the current project. I explained that being relatively early in my career I’ve been hesitant to use any kind of LLM so I can gain experience myself (to say nothing of other concerns), and yeah in his words I’ve “missed the opportunity”
Unfortunately in the majority of organizations, the idiots are at the wheels. It's not people with actual experience of how engineers do things, that dictates what those engineers should do.
Re: The threat is comfortable drift toward not understanding what you're doing
#460Earlier quoted context omitted.
How do you know you have?
I have been using it to learn Chinese along with other standard resources. My reading comprehension has improved a lot after I started to use LLMs to understand sentence structures and grammar.
Actually, I think this is a case where LLMS _can_ be useful. If we're prompting for small enough outputs, for examples around things we can already sort of reason about it, we're able to judge whether or not what's presented to use makes sense.
Presumably you're also reading some kind of learning text about the Chinese language, so the sole source isn't just the LLM?
In my experience, asking an LLM to produce small examples of well-known things (or rather, things that are going to be talked about frequently in the training data, so generally basic or fundamental topics) tend to work fine, and is going to be at a level where you yourself can judge what's presented.
I think the real danger is when a person is prompting things they don't know how to verify for themselves, since then we're basically just rolling dice and hoping