Live data from Hacker News

ChatGPT-4 significantly increased performance of business consultants

d3.harvard.edu

101–110 of 282 posts

Re: ChatGPT-4 significantly increased performance of business consultants

#101

Earlier quoted context omitted.

I’m starting to think there’s an LLM equivalent to the old saying about how everything the media writes is accurate except on the topics you’re an expert in. All LLM output looks to be good quality except when it’s output you’re an expert in. People who have no background in writing or editing think LLMs will revolutionize those fields. Actual writers and editors take one look at LLM output and can see it’s basically…

LLMs are not worthless for programming. You just cannot expect it to ship a full programm for you, but for generating functions with limited scope, I found it very useful. How to make use of a new and common libary for example. But of course you have to check and test. And for text I know people who use it succesfully (professionally) to generate texts for them as a summary from some data. They still have to proof re…

>LLMs are not worthless for programming.

They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards.

For an example. I've used Gpt4 as a sort of Google on steroids with prompts like "do subnets in gcloud span azs" and ", "in gcloud secret manager can you access secrets across regions". I very quickly learned to ask "is it true" after every answer and to never rely on a given answer too much(verify it quickly, don't let misinformation get you too far down the wrong route). So is it useful? Yes, but can it lead you down the wrong path? It very well can. The least experience you have in the field the easier it will happen.

>You just cannot expect it to ship a full programm for you, but for generating functions with limited scope, I found it very useful

Entire functions? Wow. I found it useful for generating skeletons I then have to fill by hand or tweak. I don't think I ever got anything out of Gpt4 that is useful as is (maybe except short snippets 3 lines long).

However, I found it extremely useful in parsing emails received from people or writing nice sounding replies. For that it is really good (in English).

Re: ChatGPT-4 significantly increased performance of business consultants

#102

More details in this blog post by a Wharton professor: https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the... My questions to naysayers: * Do you or anyone you know use GPT-4 (not the free GPT-3.5) to do productive tasks like coding and found it to help in many cases? * If you insist it’s useless, why do millions of people pay $20 a month to access GPT-4 and plugins?

Yes, GPT-4 is great for doing “boring work” and allows me to focus on the “fun work”. You still need to know what you’re doing though, you can’t blindly copy and paste. And for the second one, although I am paying for it too, this idea is more or less flawed nowadays. Utilization is a very hand wavy thing when it comes to this stuff. Like a purse, millions would pay money for it, some even pay thousands. But I have n…

> You still need to know what you’re doing though, you can’t blindly copy and paste.

Agreed.

> Like a purse, millions would pay money for it, some even pay thousands.

Expensive purses have intangible value for some. They are often bought to signal social status.

I'm pretty sure a significant portion of ChatGPT Plus subscribers are paying because it can help them with information or cognitive work that some people value.

Re: ChatGPT-4 significantly increased performance of business consultants

#103

Two things mentioned in the abstract that are worth pointing out. > For each one of a set of 18 realistic consulting tasks within the frontier of AI capabilities They specifically picked tasks that GPT-4 was capable of doing. GPT-4 could not do many tasks, so when we say that performance was significantly increased this is only for tasks GPT-4 is well suited to . There is still value here but let's put these results…

Hmmm. Perhaps below-average performers are more likely to take GPT output at face-value, being less competent to review and edit it. And above-average performers are more likely to hack the GPT output around, because they're confident in their own abilities.

Therefore below-average types will produce finished output more quickly; and this was a time-constrained test, so velocity matters.

ChatGPT is very good at waffling, and marketing-speak and inspirational messages are essentially waffle. IOW, the tasks were tailor-made for unaided ChatGPT, so high-performers were penalized.

Re: ChatGPT-4 significantly increased performance of business consultants

#104

Well, this sounds like perfect tasks for GPT: "Participants responded to a total of 18 tasks (or as many as they could within the given time frame). These tasks spanned various domains. Specifically, they can be categorized into four types: creativity (e.g., “Propose at least 10 ideas for a new shoe targeting an underserved market or sport.”), analytical thinking (e.g., “Segment the footwear industry market based on…

1) Your ideas are bad. 2) Spreadsheets exist. 3) No-one cares about your marketing copy. 4) No-one finds your c-suite babble inspirational. This is almost perfect input to an LLM exactly because of how low value it is in the first place.

Ha it's fun to dunk on management consultants but I think the magic is they are like pop music producers.

Somehow they're able to make the C suite hoover up LLM shovelware the same way top producers can take super obvious music and sell I V vi IV but when we try the same chords it's uninspired and no one wants to listen to it

Re: ChatGPT-4 significantly increased performance of business consultants

#105
post #33

Not surprised. It's frighteningly good, and a perfect match for programming. I often ask GPT4 to write code for something, and try if it works, but I seldom copy and paste the code it writes - I rewrite it myself to fit into the context of the codebase. But it saves me a lot of time when I am unsure about how to do something. Other times I don't like the suggestion at all, but that's useful as well, as it often clari…

The published article is not at all about programming tasks but about generating text for "strategy consultant". Some example found page 10 of the original article: - Propose at least 10 ideas for a new shoe targeting an underserved market or sport. - Segment the footwear industry market based on users. - Draft a press release marketing copy for your product. - Pen an inspirational memo to employees detailing why you…

> Nothing of real value imho.

Without the right target market, business model, and effective methods to reach customers, the most brilliant pair of shoes or piece of code can be useless (unless someone works to repurpose them as art or a teaching tool).

Re: ChatGPT-4 significantly increased performance of business consultants

#106
Can confirm. I popped the 20 bucks for GPT4, and have been using it more and more, every day for 3 weeks. Not sure how I can get by without it now. It's just so easy to have normal conversation and get answers. Like having an expert friend across the hall you can just shoutout questions, and ask for simple reminders, recommendations.

Who cares if it gets things wrong sometimes, you would double check your co-workers answers also. And there are times when I insist I am correct, and GPT will argue back and eventually I find I was wrong.

Re: ChatGPT-4 significantly increased performance of business consultants

#107
post #62
post #59

Earlier quoted context omitted.

Man, write drunk, but edit sober.

My apologies. I agree. How long does it take for Ketamine to wear off? I only had a little bit after breakfast. I hope the edit button stays.

Your horse tranquilizer-addled coworker seems to be expressing a few points about the workplace dynamics between engineers and managers. First, he believes that while there may be exceptional engineers who are also good with people, these engineers are generally not interested in managerial responsibilities. Second, he observes a communication gap between engineers and managers, where both parties avoid taking on additional tasks outside their job descriptions. Lastly, he argues that management consultants bridge this gap by identifying issues neither party wants to handle but should. He concludes by saying that he'd make a good management consultant because he can spot numerous ways a company could improve.

If all else fails, the LLM revolution will at least allow us to make sense of ketamine-induced rants on management.

Re: ChatGPT-4 significantly increased performance of business consultants

#108

Earlier quoted context omitted.

LLMs are not worthless for programming. You just cannot expect it to ship a full programm for you, but for generating functions with limited scope, I found it very useful. How to make use of a new and common libary for example. But of course you have to check and test. And for text I know people who use it succesfully (professionally) to generate texts for them as a summary from some data. They still have to proof re…

>LLMs are not worthless for programming. They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards. For an example. I've used Gpt4 as a sort of Google on steroids with prompts like "do subnets in gcloud span azs" and ", "in gcloud secret manager can you access secrets across regions". I very quickly learned to ask "is it true" after every answer…

> They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards.

I basically gave up on llms because i was spending more time figuring out what it did wrong than actually getting value.

People without programming skill are still impressed by them. But they yet have to learn or deliver anything of value even with the help of chat bots.

Re: ChatGPT-4 significantly increased performance of business consultants

#109

Earlier quoted context omitted.

1) Your ideas are bad. 2) Spreadsheets exist. 3) No-one cares about your marketing copy. 4) No-one finds your c-suite babble inspirational. This is almost perfect input to an LLM exactly because of how low value it is in the first place.

Ha it's fun to dunk on management consultants but I think the magic is they are like pop music producers. Somehow they're able to make the C suite hoover up LLM shovelware the same way top producers can take super obvious music and sell I V vi IV but when we try the same chords it's uninspired and no one wants to listen to it

Or, you have a break out hit like the Spice Girls.

If it was so bad, then why do people listen?

There is still a market.

Does the market suck? Full of idiots?

Your argument ends up being that successful things are bad, because humans are just idiots and thus if something is successful it is because it is just liked by idiots.

As much as I might agree generally, it doesn't get you far.

Re: ChatGPT-4 significantly increased performance of business consultants

#110

> Two distinct patterns of AI use emerged: “Centaurs,” who divided and delegated tasks between themselves and the AI, and “Cyborgs,” who integrated their workflow with the AI. It was nice of them to explain that the article was total horse dung before having to read the whole paper

Centuar is a term of art in the chess community: https://historyofinformation.com/detail.php?entryid=4724
Post reply on HN