Live data from Hacker News

Workers with less experience gain the most from generative AI

mitsloan.mit.edu

101–110 of 120 posts

Re: Workers with less experience gain the most from generative AI

#101
post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The fact that you can't trust it makes it a problem for novices. My view has been that it can help experts remove some of the mundane stuff that takes time but needs to get done. You can think of it as an assistant you always need to check on but since you are an expert you can recognize when there is a problem.

Lately, I've revised my view a bit. AI can get novices started on getting a project done and it can be a source of information for the project. Googling works but I think it's much faster querying an LLM. It's good for certain types of projects but I wouldn't trust it for important projects unless I can double check but the double checking will make it a drag to use. I think LLMs do have a place for novices but you have to be careful.

Re: Workers with less experience gain the most from generative AI

#102

This is exactly the opposite of my experience with it. Getting it to code, it can save me, a fairly experienced coder, quite a lot of time because it can do some boring things - boilerplate stuff, or things I can’t be bothered to look up. And I can tell if it’s done it right or not. I’ve seen people with less coding experience use it and just take its suggestions at face value with painful results and definite detrim…

My experience is similar. It’s a multiplier. The more capable you already are, the more you can get out of it. Perhaps not surprising in that all tools work this way.

Re: Workers with less experience gain the most from generative AI

#104
post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information. People that are 5-10x more effective at hard skills like programming, well I'm just…

some people are not programmer butneed to write small script in work or even just finding a Excel function.

Re: Workers with less experience gain the most from generative AI

#105

Earlier quoted context omitted.

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information. People that are 5-10x more effective at hard skills like programming, well I'm just…

I do get a big boost from GPT-4 in some areas, for a different reason - it's a great and versatile tool for overcoming random mental and emotional barriers. There are plenty of tasks I'd normally procrastinate on, or be reluctant to do, because they're tiring, boring, or emotionally difficult for reasons specific to myself. That is, cases when I have the knowledge and the skills, but lack the willpower or composture…

Ye, sometimes I don't have enough energy to research how to implement something that day, so I'd put it off till I had the energy. Instead, gpt generates the code, and I just have to debug/test it.

Re: Workers with less experience gain the most from generative AI

#106
post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

i tried chatgpt to produce some ansible playbooks for postgresql_privs module - that module uses or used to use rather convoluted logic and i could never exactly wrap my head around that one ( and i had a rather non-trivial use case for it, so maybe that contributed to my struggles).

the produced playbooks passed the syntax check, that much i can admit. it went downhill from there, producing files with completely invalid keywords.

Re: Workers with less experience gain the most from generative AI

#107
post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information. People that are 5-10x more effective at hard skills like programming, well I'm just…

> Basically, anyone that says that is telling on themselves.

Similar to anyone who says, “When you'd have a hard time googling something, you ask the LLM.”

Just my opinion.

Re: Workers with less experience gain the most from generative AI

#108
post #74

Earlier quoted context omitted.

Have you tried this at all? I find that many of the generative AI models (GPT-4, 3.5, even MPT-30B running on my laptop) are really shockingly good at SQL. Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. Generating new SQL queries is a bit harder, because of the context you need to provide - but I've had very strong results from that as well.…

> Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. [...] If you've found differently I'd love to hear about it. I have not directly tried it (the employer does not allow AI chatbots for any application intended for production (i.e. more sensitive stuff), but only for doing experiments), but working on the code I very rarely had the problem that…

Those two questions require a whole bunch of additional context. Has it ever been written down anywhere, or does it exist only in the heads of members of staff who understand those decisions? If the latter then yeah, there's clearly no way an AI could ever guess those things.

Part of the trick of making good use of LLMs is having a good instinct as to what kind of questions they will be able to ask and what kind of questions they are likely to mess up.

As a quick demo, I threw quite a gnarly real-world SQL query from this notebook - https://observablehq.com/@simonw/blog-to-newsletter#sql - into GPT-4 just now and gave it the following prompt:

> Explain this SQL query four times: first, as a high level explanation of what it does. Secondly, as some pithy highlights as to clever tricks it uses. Thirdly, as a step-by-step guide to exactly what each piece of the query does. Finally, provide some advice on how the query could be improved.

Here's what it produced in response: https://chat.openai.com/share/47b2f1c5-6b88-4ff2-b6d1-8f58da...

The high level explanation is, I think, excellent. I wrote that query and I'd forgotten the detail about how it sorts with entries first.

The tips for improvements are the weakest part, since they make assumptions that don't hold for this particular query (for one thing, SQLite doesn't have the ability to run stored procedures).

Re: Workers with less experience gain the most from generative AI

#109

Earlier quoted context omitted.

I dunno computers should be world changing. And they are. Yet the lives of ordinary people have hardly improved and are arguably worse since the 1980s

> I dunno computers should be world changing. And they are. Yet the lives of ordinary people have hardly improved and are arguably worse since the 1980s If you replace "lives of ordinary people" by "productivity improvement", you actually have a good (and rarely discussed) point: > https://en.wikipedia.org/wiki/Productivity_paradox

On that note, my personal hypothesis, even more controversial than "IT unproductive hypotheses" in the article that means IT was a rounding error relative to earlier major improvements, is: in many areas - particularly in office work and all kinds of everyday errands - IT is anti-productive, as in it makes people less productive on the net.

The core hypothesis behind my belief is that introduction of computers to replace a class of tasks - up to and including a whole job type - is just shifting the workload, diffusing it across many people, where previously it was concentrated in smaller number of specialists. Think e.g. the things you use Word, Excel, Powerpoint, Outlook, etc. (or their equivalents from other vendors) for - before software ate it, a lot of that used to be someone's job. Now, it's just tacked onto everyone's workload, distracting people from doing the actual job they were paid to.

That would seem like obviously stupid way to do, so why would businesses all fall for it? I argue it's because even as shifting the workload makes everyone in the company less productive on the net, it looks like an improvement to accounting. Jobs with salaries are legible, clearly visible on the balance sheets. So is money saved by eliminating them. However, the overall productivity drop caused by smearing that same work across rest of the company? That's incremental, not obviously quantifiable. People and their salaries stay the same. So it all looks like introducing software and obsoleting some jobs saves everyone money - but then somehow, everyone is experiencing a "productivity paradox". But it's not a paradox if you ignore the financial metrics with their low resolution - focusing on what happens to work, it seems that IT improvements are mostly a lie.

Re: Workers with less experience gain the most from generative AI

#110
In my personal experience chatgpt-4 has been a great addition to the toolbox, both for coding assistance and research. You do have to have sufficient grounding in the subjects to be able to evaluate the responses critically though.

However, in all cases I have seen a very substantial decline of the capabilities of chatgpt-4 with the last few releases. E.g. it used to get code snippets most often right before. Whereas now it tends to be wrong most of the time. Usually it conflates capabilities of several distict libraries, just hallucinating (extrapolating) some non existing functions or attributes.

I personally suspect they are 'cleansing' the training data, and/or driving severe 'schizophrenia' into the model through conflicting RLHF.

Post reply on HN