Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Workers with less experience gain the most from generative AI

mitsloan.mit.edu

Post reply on HN
Print view View original

81–90 of 120 posts

Re: Workers with less experience gain the most from generative AI

#81

Post by Apfel »

ren_engineer wrote: post #41
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

as always the question is are you using GPT4? Almost all people downplaying it are using free ChatGPT accounts and Bard is still a joke at this point GitHub Copilot is also very good because it has the context of your open files or selected text and will pretty regularly give you exactly what you want for the next line of code. I will say it's much weaker for less popular libraries and programming languages, but for…

Have been using gpt4 and just copying all the relevant files in with something along the lines of

``` Given these X files, help me assess how best to implement feature Y. Ask any clarifying questions that may assist with writing a best practice solution. *File 1* *File 2* *File 3* ... ```

If the context length is a problem I'll use claude-100k.

Pretty much always results in good quality code.

I'll then typically open a new window context and get it to review according to best practices highlighting missing e.g. error handling or possible edge cases.

Re: Workers with less experience gain the most from generative AI

#82

Post by siliconc0w »

Currently, it's basically entry level for anything mildly complicated and it can't troubleshoot itself, so it needs to be coached into giving the right answer. So it's great for boilerplate and API lookups but if you were using a typed language before and had a decent IDE, this was already a solved problem. It seems impressive because it's very fast but that advantage goes away when you then have to be like, "hmm, well that's not quite right" and then you coach it into being correct (or close enough that you stop).

If your team can't autocomplete APIs, and if you can't run everything locally so you can step through your code with a debugger, and if you don't have good tests so you need an AI to tell when you broke something (see also profiling, telemetry, release quickly and confidently, etc) - you're going to get a million percent more mileage just doing these sort of basics correctly (and fast) vs investing in AI because the open secret is that most of engineering time is spent reading and debugging code rather than writing it. This should be table stakes but so many engineering teams (even at bigtech companies) can't do the above. So they're all signing up for copilot but they still iterate with logging statements and catch regressions in production.

Re: Workers with less experience gain the most from generative AI

#83

Post by hmage »

Lifehack, add this to the end of your question:

  Start your response with "Let's work this out in a step by step way to be sure we have the right answer:" and work this out in a step by step way to be sure you have the right answer
It will improve your answers significantly (for example, on word math problems, solve rate went from 18% to 79%).

Source: https://github.com/openai/openai-cookbook/blob/main/techniqu...

Re: Workers with less experience gain the most from generative AI

#84

Post by waldohatesyou »

dmarchand90 wrote: post #47
JimtheCoder wrote: post #32

Earlier quoted context omitted.

Worse in what sense? Computers have done amazing things for humanity. I would not want to go back to a computerless world...

If you work in the US for fang or similar your life is probably much better. I love computers. I love what they do for recreation and keeping in touch long distance. But work hours are longer. Education and housing is no longer affordable for most people. In the US life expectancy is beginning to drop. You would think people would have easier less stressful lives with greater financial security. But it's not the case…

I think the catch here is that we need to know what makes admin work increase. While I can’t comment on education, at least for housing in Canada this is all a result of NIMBYism and government intervention. In other words, we could have slouched towards utopia much further than we have if it were not for NIMBYs.

Re: Workers with less experience gain the most from generative AI

#85

Post by barrkel »

somishere wrote: post #68

Reading the comments here made me think of a dishwashing analogy. I grew up without a dishwasher (when we asked our parents why we didn't have one my dad responded, "what do you mean? We have six!" referring to us kids). As an adult we have one in the house but it is rarely used. Washing dishes is a menial task but I enjoy it. It's hard to articulate exactly why. Needless to say I'm quite an experienced dish washer,…

If you have 10-15% of items with food stuck on them after a wash, you need either a better dishwasher, better detergent (there is a noticeable range in quality) or both.

Re: Workers with less experience gain the most from generative AI

#86

Post by JimtheCoder »

waldohatesyou wrote: post #84
dmarchand90 wrote: post #47

Earlier quoted context omitted.

If you work in the US for fang or similar your life is probably much better. I love computers. I love what they do for recreation and keeping in touch long distance. But work hours are longer. Education and housing is no longer affordable for most people. In the US life expectancy is beginning to drop. You would think people would have easier less stressful lives with greater financial security. But it's not the case…

I think the catch here is that we need to know what makes admin work increase. While I can’t comment on education, at least for housing in Canada this is all a result of NIMBYism and government intervention. In other words, we could have slouched towards utopia much further than we have if it were not for NIMBYs.

"In other words, we could have slouched towards utopia much further than we have if it were not for NIMBYs."

I think the NIMBYs have done pretty well in creating their own Utopia in terms of Canadian real estate

Re: Workers with less experience gain the most from generative AI

#87

Post by revolvingocelot »

larve wrote: post #42
pm_me_your_quan wrote: post #26

Earlier quoted context omitted.

I'd be really curious if you're willing to expand more on how it has helped with those workflows. Do you copy/paste chunks in and ask it to explain them? Have it try to refactor them and then clean up?

For legacy code: - generate comments (hit or miss, but at least it can rewrite my random notes into consistent notes) - generate type annotations - refactor "broadly" (say, "rename all variables to match the following style" or "turn this class into a dataclass like XXX" or "transform the SQL queries into builder queries using XYZ"). Often requires some manual work but it gets a lot of tedious stuff out of the way -…

>- refactor "broadly" (say, "rename all variables to match the following style" or "turn this class into a dataclass like XXX" or "transform the SQL queries into builder queries using XYZ"). Often requires some manual work but it gets a lot of tedious stuff out of the way

Can you go on more about this, please? This sounds, frankly, heavenly, but the second sentence gives me pause. I guess it's not necessarily a question of how reliably it can "broadly" refactor but rather how broadly "broadly" is meant to be taken...

>- generate SQL queries at the speed of mouth

...and this? I'm not really a database guy, but I do keep hearing from them about how (eg, a database guy's) stateful knowledge of a database can result much, much more efficient queries than eg a sales guy with a query builder. Are the robut's queries more like the former or the latter?

Re: Workers with less experience gain the most from generative AI

#89

Post by aleph_minus_one »

simonw wrote: post #74
aleph_minus_one wrote: post #60

Earlier quoted context omitted.

> - generate SQL queries at the speed of mouth Because of the points this is the nearest to the work that some colleagues do, I anakyze this point (but you could ask similar questions about many of the other points): In my experience, writing correct SQL queries (which often tend to be quite non-trivial because of the internal complexity of the projects) typically involves a lot of knowledge about the whole system th…

Have you tried this at all? I find that many of the generative AI models (GPT-4, 3.5, even MPT-30B running on my laptop) are really shockingly good at SQL. Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. Generating new SQL queries is a bit harder, because of the context you need to provide - but I've had very strong results from that as well.…

> Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. [...] If you've found differently I'd love to hear about it.

I have not directly tried it (the employer does not allow AI chatbots for any application intended for production (i.e. more sensitive stuff), but only for doing experiments), but working on the code I very rarely had the problem that I could not understand what some single (SQL) line of code does in the "programming sense".

The central problem that rather occurs often is understanding why this line does exist and why things are implemented the way they are.

Just to give an example: to accelerate some queries, I thought some index would make sense (colleagues principally agreed; it would likely accelerate a particular query that I had in mind). But there exists a good reason why there exists no index at this table (as the respective colleague explained to me). This again implies that for ETL stuff involving particular tables, one should make use of temporary tables where possible instead of JOINs; this is the reason why the code is organized as it is. This is the kind of explanation that I need, which surely no AI can deliver.

Or another example: why does some particular function (1) have a rights check for a "more powerful" role and a related one (2) does not need one? The reason is very interesting: principally having this check (for a "more powerful" role) does not make a lot of sense, but for some very red-tape reasons auditors requested that only a particular group of roles shall be allowed to execute (1), but they were perfectly fine with a much larger group of users being allowed to execute (2). Again something that no AI will be able to answer.

Re: Workers with less experience gain the most from generative AI

#90

Post by TeMPOraL »

godelski wrote: post #34
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information. People that are 5-10x more effective at hard skills like programming, well I'm just…

I do get a big boost from GPT-4 in some areas, for a different reason - it's a great and versatile tool for overcoming random mental and emotional barriers.

There are plenty of tasks I'd normally procrastinate on, or be reluctant to do, because they're tiring, boring, or emotionally difficult for reasons specific to myself. That is, cases when I have the knowledge and the skills, but lack the willpower or composture (or glucose / caffeine in my bloodstream). Using GPT-4 for with that kind of work isn't saving much time vs. what I could do, but it is compared to what would actually happen, which is either me procrastinating on it, delaying it for a better time (next morning, day with less meetings, etc.), or suffering a 2-10x performance penalty from having to fight through my own emotional blocks.

On the net, this isn't making me 5-10x more effective at work. It's probably not even 2x, short-term. Mid-term, 2-5x would be possible, because all the things I did earlier than later add up. Time will tell.

Am I telling on myself here? Maybe. Sorry not sorry. I am a human being, with a human brain, which means some things that should be easy for me, become hard for unrelated reasons. GPT-4 is one of many tools I have to overcome such challenges, but it's a particularly powerful and versatile one, so I'm happy that I can use it.

(Also, INB4, I have access to company-approved deployment on Azure, so I'm in the clear with using it at work.)

Post reply on HN