Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT-4 significantly increased performance of business consultants

d3.harvard.edu

Post reply on HN
Print view View original

31–40 of 282 posts

Re: ChatGPT-4 significantly increased performance of business consultants

#31

Post by highwaylights »

hubraumhugo wrote: post #15

Well, this sounds like perfect tasks for GPT: "Participants responded to a total of 18 tasks (or as many as they could within the given time frame). These tasks spanned various domains. Specifically, they can be categorized into four types: creativity (e.g., “Propose at least 10 ideas for a new shoe targeting an underserved market or sport.”), analytical thinking (e.g., “Segment the footwear industry market based on…

1) Your ideas are bad.

2) Spreadsheets exist.

3) No-one cares about your marketing copy.

4) No-one finds your c-suite babble inspirational.

This is almost perfect input to an LLM exactly because of how low value it is in the first place.

Re: ChatGPT-4 significantly increased performance of business consultants

#32

Post by skepticATX »

Two things mentioned in the abstract that are worth pointing out.

> For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities

They specifically picked tasks that GPT-4 was capable of doing. GPT-4 could not do many tasks, so when we say that performance was significantly increased this is only for tasks GPT-4 is well suited to. There is still value here but let's put these results into context.

> Consultants across the skills distribution benefited significantly from having AI augmentation, with those below the average performance threshold increasing by 43% and those above increasing by 17% compared to their own scores

Even when cherry-picking tasks that GPT-4 is particularly suited for, above average performers only increased performance by 17%. This increase is still impressive, were it to be seen across the board. But I do think that 17% is a lot less than some people are trying to sell.

Re: ChatGPT-4 significantly increased performance of business consultants

#33

Post by Rexxar »

awestroke wrote: post #5

Not surprised. It's frighteningly good, and a perfect match for programming. I often ask GPT4 to write code for something, and try if it works, but I seldom copy and paste the code it writes - I rewrite it myself to fit into the context of the codebase. But it saves me a lot of time when I am unsure about how to do something. Other times I don't like the suggestion at all, but that's useful as well, as it often clari…

The published article is not at all about programming tasks but about generating text for "strategy consultant".

Some example found page 10 of the original article:

   - Propose at least 10 ideas for a new shoe targeting an underserved market or sport.
   - Segment the footwear industry market based on users.
   - Draft a press release marketing copy for your product.
   - Pen an inspirational memo to employees detailing why your product would outshine competitors.
Nothing of real value imho.

Re: ChatGPT-4 significantly increased performance of business consultants

#34

Post by awestroke »

dkjaudyeqooe wrote: post #29
bryancoxwell wrote: post #9

Earlier quoted context omitted.

I’ve also found the act of describing my problem to GPT4 is sometimes just a helpful as the answer itself. It’s almost like enhanced rubber duck debugging.

We need an inverse GPT4-style LLM that doesn't provide answers but instead asks relevant questions.

GPT4 can do that too. Just show it something (code or text) and ask it to ask coaching questions about it.

Re: ChatGPT-4 significantly increased performance of business consultants

#35

Post by catchnear4321 »

dopylitty wrote: post #22
mgaunard wrote: post #2

Says more about how useless BCG consultants are.

I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically…

everyone you described share something in common.

they aren’t good at using language models.

Re: ChatGPT-4 significantly increased performance of business consultants

#36

Post by awestroke »

dkjaudyeqooe wrote: post #23
awestroke wrote: post #5

Not surprised. It's frighteningly good, and a perfect match for programming. I often ask GPT4 to write code for something, and try if it works, but I seldom copy and paste the code it writes - I rewrite it myself to fit into the context of the codebase. But it saves me a lot of time when I am unsure about how to do something. Other times I don't like the suggestion at all, but that's useful as well, as it often clari…

This is one step removed from "try different things until it works" style of programming. Not to say you're of of those programmers, but it certainly enables those sorts of programmers.

And what's the harm in that? That's how I first started out programming decades ago.

Re: ChatGPT-4 significantly increased performance of business consultants

#37

Post by mikro2nd »

catchnear4321 wrote: post #35
dopylitty wrote: post #22

Earlier quoted context omitted.

I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically…

everyone you described share something in common. they aren’t good at using language models.

Nor are 99.9% of humanity. I think that's the point.

Re: ChatGPT-4 significantly increased performance of business consultants

#38

Post by xbmcuser »

There is a lot of office work that will overtime be optimized over time using gpt like services. I was tech savvy enough to know that a lot of office work that I do is repeatable and can be done using scripts but not good enough to write those scripts myself. Using Chat gpt allowed me to write those scripts it took me I think 15-20hrs to get the scripts working perfectly. I knew just a little bit of python scripting did not know anything about python pandas or xls writer etc but was able to create something that saves me I would estimate 20-25 hours a week.

In my opinion a lot of people here on hackernews as they are themselves good at programing underestimate how services like chat gpt can open a new world to non programmers. They also probably make the non inquisitive learn less. Previously to learn how to stop multiple snapd services using a script I would have googled and then cobbled together something today I just ask chatgpt and get a working script in less than a min.

Re: ChatGPT-4 significantly increased performance of business consultants

#39

Post by skybrian »

dopylitty wrote: post #22
mgaunard wrote: post #2

Says more about how useless BCG consultants are.

I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically…

Programmers don't think that, though, or least not all the time.

You could say similar things about Stack Overflow, and yet we use it.

Post reply on HN