Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Workers with less experience gain the most from generative AI

mitsloan.mit.edu

Post reply on HN
Print view View original

91–100 of 120 posts

Re: Workers with less experience gain the most from generative AI

#91

Post by harrisi »

somishere wrote: post #68

Reading the comments here made me think of a dishwashing analogy. I grew up without a dishwasher (when we asked our parents why we didn't have one my dad responded, "what do you mean? We have six!" referring to us kids). As an adult we have one in the house but it is rarely used. Washing dishes is a menial task but I enjoy it. It's hard to articulate exactly why. Needless to say I'm quite an experienced dish washer,…

I've gone back and forth with dishwashers my whole life, spending years with followed by years without, etc. and I finally am team dishwasher. There are a few tips I've learned, such as immediately rinsing a dish after using (I don't bother scrubbing), running the hot water for a minute before starting, using the pre-wash detergent and shine stuff appropriately. I also don't bother entirely filling it up to run it - which may seem wasteful, but I'd imagine filling it completely and having to rewash some percent of them ends up being more wasteful.

Also, just like getting a manual dishwashing routine figured out, dishwashers are all different and need to be learned like any tool. Understanding where pieces are, which cycles run when, which compartments open and close and even how they do it, are all important. It really is best to think of it as a tool.

Some useful videos:

https://youtu.be/_rBO8neWw04 https://youtu.be/Ll6-eGDpimU

Re: Workers with less experience gain the most from generative AI

#92

Post by copperx »

aleph_minus_one wrote: post #89
simonw wrote: post #74

Earlier quoted context omitted.

Have you tried this at all? I find that many of the generative AI models (GPT-4, 3.5, even MPT-30B running on my laptop) are really shockingly good at SQL. Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. Generating new SQL queries is a bit harder, because of the context you need to provide - but I've had very strong results from that as well.…

> Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. [...] If you've found differently I'd love to hear about it. I have not directly tried it (the employer does not allow AI chatbots for any application intended for production (i.e. more sensitive stuff), but only for doing experiments), but working on the code I very rarely had the problem that…

> This is the kind of explanation that I need, which surely no AI can deliver.

But have you tried? GPT-4 is great for coming up with hypotheses of why thing are the way they are.

Re: Workers with less experience gain the most from generative AI

#93

Post by godelski »

TeMPOraL wrote: post #90
godelski wrote: post #34

Earlier quoted context omitted.

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information. People that are 5-10x more effective at hard skills like programming, well I'm just…

I do get a big boost from GPT-4 in some areas, for a different reason - it's a great and versatile tool for overcoming random mental and emotional barriers. There are plenty of tasks I'd normally procrastinate on, or be reluctant to do, because they're tiring, boring, or emotionally difficult for reasons specific to myself. That is, cases when I have the knowledge and the skills, but lack the willpower or composture…

I guess I'm a bit confused at what you're saying. Are you asking it to explain things to you in a pirate voice so that it is more entertaining and you can maintain concentration? I don't find this as really relying on GPT. Or are you saying when you have simple routines that need to be implemented but are boring and so you have GPT generate most of them for you to reduce the bordem? I also don't see that as contentious with what I said above (all programming has a lot of boring and routine shit).

But if you're outsourcing a significant portion of your overall work (day-to-day, not just off-days) then that's more what I'm getting at. The people that are like "I couldn't imagine programming without copilot" or such. Every one of those that I've met is missing important base knowledge about programming in general that ends up making a lot of technical debt for themselves. The "build fast and break things" style realistically only works if you have a certain level of expertise OR the project doesn't need to be robust. Former because to actually be fast you need to have a good picture of the whole or else you end up chasing one thing to the next and your overall path is far slower but might seem faster because you're sprinting the whole time.

Re: Workers with less experience gain the most from generative AI

#94

Post by angrais »

aleph_minus_one wrote: post #89
simonw wrote: post #74

Earlier quoted context omitted.

Have you tried this at all? I find that many of the generative AI models (GPT-4, 3.5, even MPT-30B running on my laptop) are really shockingly good at SQL. Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. Generating new SQL queries is a bit harder, because of the context you need to provide - but I've had very strong results from that as well.…

> Paste in a query and ask it for a detailed explanation. I've genuinely not seen it NOT provide a good result for that yet. [...] If you've found differently I'd love to hear about it. I have not directly tried it (the employer does not allow AI chatbots for any application intended for production (i.e. more sensitive stuff), but only for doing experiments), but working on the code I very rarely had the problem that…

Write some fake table. Paste it into ChatGPT, then come back to the discussion.

So far you've said that things like "This is the kind of explanation that I need, which surely no AI can deliver." but have not actually tried the system?

As GP asked: have you tried ChatGPT or similar LLMs? If not, go do it .. you may be surprised.

Re: Workers with less experience gain the most from generative AI

#95

Post by hackerlight »

godelski wrote: post #34
camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

The thing I find them best for is fuzzy searching. When you'd have a hard time googling something, you ask the LLM. The answer you get back might also be fuzzy, but often that can be defuzzed and then correctly googled or resourced. In this way it is highly effective. But that is reducing the amount of time searching for information. People that are 5-10x more effective at hard skills like programming, well I'm just…

> People that are 5-10x more effective at hard skills like programming, well I'm just convinced they weren't a good programmer to begin with and are doing easy problems.

It can make devs 5-10x more effective at certain self-contained programming tasks, even if they aren't 5-10x more effective at their job overall. If you have to write a script in a language you haven't used before and it needs to interface with an API whose documentation you haven't read, and the script isn't mission critical but a "nice to have", GPT can massively shorten the learning curve required to just get this work done. Being a good dev doesn't excuse you from having to pay the cost of learning the syntax of this language and basics of the documentation more thoroughly than if you didn't have GPT.

Re: Workers with less experience gain the most from generative AI

#96

Post by diob »

camhart wrote: post #6

My experience with Bard/ChatGPT has been that you'll easily shoot your foot off if you don't know what you're doing. Dangerous for workers with less experience. I see claims that it makes you 7-8x more effective. That hasn't been my experience. Maybe 5-10% improvement at best. Instead of Googling myself, LLMs can sometimes give me the answer more quickly than I could find it. Once you ask it anything not easily scrap…

For me it automates the low level stuff so I can focus on other things.

"Given this json blob what is the typescript type?" "Given this type, make a function that loops over X and makes a map of Y?"

Or I use it to write code in languages I'm not familiar with. Heck I even used it to make some github actions cause I severely dislike yaml.

It did hallucinate a bit on the github actions (especially since it didn't know the latest api), but it put me on the right track. So I don't know, I think it enables me to do work in areas I normally wouldn't, like Java. It isn't always right, but it helps me immediately test ideas / iterate.

I think of it like reference I can ask questions without getting sass or having to worry about some toxic coworker wondering why I don't just know "x". It's not always right, but neither are my coworkers / humans. So I guess, treat it like a human aka fallible?

Re: Workers with less experience gain the most from generative AI

#97

Post by larve »

aleph_minus_one wrote: post #60
larve wrote: post #42

Earlier quoted context omitted.

For legacy code: - generate comments (hit or miss, but at least it can rewrite my random notes into consistent notes) - generate type annotations - refactor "broadly" (say, "rename all variables to match the following style" or "turn this class into a dataclass like XXX" or "transform the SQL queries into builder queries using XYZ"). Often requires some manual work but it gets a lot of tedious stuff out of the way -…

> - generate SQL queries at the speed of mouth Because of the points this is the nearest to the work that some colleagues do, I anakyze this point (but you could ask similar questions about many of the other points): In my experience, writing correct SQL queries (which often tend to be quite non-trivial because of the internal complexity of the projects) typically involves a lot of knowledge about the whole system th…

I approach things a bit obliquely. I create a custom made DSL (starting from scratch in each conversation, often) that allows me to model my query the way I want. Then, I write a traditional SQL builder on that DSL (or more like, ask GPT to do it for me). Then, I generate DSL statements that match my current domain, and more importantly, modify existing ones.

So, at each step, I do almost trivial transformations.

One key ingredient is that the DSL should include many "description" fields that incorporate english language, because that helps the model "understand" what the terser DSL fields are for.

Straight SQL is a crapshoot, and as you said, more often than not, either obviously or subtly broken or for another database. Which makes sense, considering how much different flavors of SQL it has in its training corpus and how much crappy SQL is out there anyway.

Another thing that helps is use extremely specific "jargon" for the domain you want to write queries for. Asking for "accrual revenue" and "yoy avg customer value" (yes, yoy, not year over year) often tends to bring back much higher quality than just asking for "revenue" or "customer value".

Re: Workers with less experience gain the most from generative AI

#98

Post by somishere »

barrkel wrote: post #85
somishere wrote: post #68

Reading the comments here made me think of a dishwashing analogy. I grew up without a dishwasher (when we asked our parents why we didn't have one my dad responded, "what do you mean? We have six!" referring to us kids). As an adult we have one in the house but it is rarely used. Washing dishes is a menial task but I enjoy it. It's hard to articulate exactly why. Needless to say I'm quite an experienced dish washer,…

If you have 10-15% of items with food stuck on them after a wash, you need either a better dishwasher, better detergent (there is a noticeable range in quality) or both.

It's quite a high quality dishwasher .. multi-drawer job. But I will look into the detergent :)

Re: Workers with less experience gain the most from generative AI

#99

Post by somishere »

harrisi wrote: post #91
somishere wrote: post #68

Reading the comments here made me think of a dishwashing analogy. I grew up without a dishwasher (when we asked our parents why we didn't have one my dad responded, "what do you mean? We have six!" referring to us kids). As an adult we have one in the house but it is rarely used. Washing dishes is a menial task but I enjoy it. It's hard to articulate exactly why. Needless to say I'm quite an experienced dish washer,…

I've gone back and forth with dishwashers my whole life, spending years with followed by years without, etc. and I finally am team dishwasher. There are a few tips I've learned, such as immediately rinsing a dish after using (I don't bother scrubbing), running the hot water for a minute before starting, using the pre-wash detergent and shine stuff appropriately. I also don't bother entirely filling it up to run it -…

Thanks for the thoughtful advice. Agree on it being tool, same as LLMs. I guess my point, beyond the practical, was that there are some aspects to the tool-less experience that are hard to replace.

I'm thinking patience and, I guess, the other meta skills and behaviours that aren't directly related to the washing itself.

That said, your pro-dishwasher explanation talks to some of these too.

Re: Workers with less experience gain the most from generative AI

#100

Post by larve »

revolvingocelot wrote: post #87
larve wrote: post #42

Earlier quoted context omitted.

For legacy code: - generate comments (hit or miss, but at least it can rewrite my random notes into consistent notes) - generate type annotations - refactor "broadly" (say, "rename all variables to match the following style" or "turn this class into a dataclass like XXX" or "transform the SQL queries into builder queries using XYZ"). Often requires some manual work but it gets a lot of tedious stuff out of the way -…

>- refactor "broadly" (say, "rename all variables to match the following style" or "turn this class into a dataclass like XXX" or "transform the SQL queries into builder queries using XYZ"). Often requires some manual work but it gets a lot of tedious stuff out of the way Can you go on more about this, please? This sounds, frankly, heavenly, but the second sentence gives me pause. I guess it's not necessarily a quest…

I gave some insights on the SQL thing above. For the refactor broadly, it's useful when I have something that's a bit too squishy for my IDE refactoring tools/multicursor editing/vim macros, but easy enough to do or provide an example for. One thing I mentioned is having consistent variable names.

I would highly recommend taking a piece of code (any code) and then just start experimenting. Here's a few prompt ideas:

  - make this a singleton
  - use more classes
  - use less classes
  - create more functions
  - use lambdas
  - rewrite in a functional pipeline style
  - extract higher order types
  - use fluent APIs
  - use a query builder
  - transform to a state machine
  - make it async
  - add cancellation
  - use a work queue
  - turn it into a microservice pipeline
  - turn it into a text adventure
  - create a declarative DSL to simplify the core logic
  - list the edge cases
  - write unit tests for each edge case
  - transform the unit tests into table-driven tests
  - create a fuzzing harness
  - transform into a REST API
  - write a CLI tool
  - write a websocket server to stream updates into a graph
  - generate a HTML frontend
  - add structured logging
  - create a CPU architecture to execute this in hardware
  - create a config file
  - generate test data
  - generate a bayesian model to generate test data
  - generate a HTML frontend to generate a bayesian model to generate test data and download as a csv
  - etc...
If you are not feeling inspired, take a random computer science book, open at a random page, and literally just paste some jargon in there and see what happens. You don't need correct sentences or anything, just random words.

There really is nothing that can go wrong, in the worst case the result is gibberish. The code doesn't even need to build or be correct for it to be useful. These models are trained to be plausible, and even more importantly, self-consistent.

When prompted with code in-context, these things are amazing at figuring out consistent, plausible, elegant, mainstream APIs. Implementing them correctly is something I usually tend to do manually instead of bludgeoning the LLM.

Post reply on HN