Where ChatGPT could excel is early education learning where the ideas are simple and universally agreed and written online. As you go higher level the chance of hallucinations becomes higher and you could be taught the wrong thing without knowing the risks
ChatGPT-4 significantly increased performance of business consultants
141–150 of 282 posts
Re: ChatGPT-4 significantly increased performance of business consultants
#142Well, this sounds like perfect tasks for GPT: "Participants responded to a total of 18 tasks (or as many as they could within the given time frame). These tasks spanned various domains. Specifically, they can be categorized into four types: creativity (e.g., “Propose at least 10 ideas for a new shoe targeting an underserved market or sport.”), analytical thinking (e.g., “Segment the footwear industry market based on…
> Target: Dog Walkers
> Features: Built-in waste bag dispenser
I'm not yet sure whether I hate it or love it.
> Target: Visually Impaired Individuals
> Features: Haptic feedback
Haptic shoes, how revolutionary!
Re: ChatGPT-4 significantly increased performance of business consultants
#143LLMs are stunningly good at language tasks: almost all of what us old-timers called NLP is just crushed these days. Summarization, Q&A, sentiment, the list goes on and on. Truly remarkable stuff. And where there isn’t a bright line around “fact”, and where it doesn’t need to come together like a Pynchon novel, the generative stuff is smoking hot: short-form fiction, opinion pieces, product copy? Massive productivity…
Try asking it for specifically NLP-ized text and it does it very well...
(but then tells you not to use it as it is "unethical")
Re: ChatGPT-4 significantly increased performance of business consultants
#144My prediction? In about 6 months, every test, task, or use of a LLM for anything that requires a modicum of creativity is going to find that it only has a fixed set of "ideas" before it starts regurgitating them. [0] I can easily imagine this in their hypothetical shoe pitch question, and many models going for more factual answers have been rapidly showing this bias by design.
[0] https://www.marktechpost.com/2023/06/16/this-paper-tests-cha...
Re: ChatGPT-4 significantly increased performance of business consultants
#145LLMs are stunningly good at language tasks: almost all of what us old-timers called NLP is just crushed these days. Summarization, Q&A, sentiment, the list goes on and on. Truly remarkable stuff. And where there isn’t a bright line around “fact”, and where it doesn’t need to come together like a Pynchon novel, the generative stuff is smoking hot: short-form fiction, opinion pieces, product copy? Massive productivity…
I wonder how long it takes AI to get good at law. Right now the verbal tasks it excels at are similar to the artistic ones: namely, solving problems with enormous solution spaces that are robust to small perturbations. That is, change a good picture of an angry tree man slightly and it's still probably a good picture of an angry tree man.
I've tried using a lot for writing motions. It can actually do a pretty decent job of writing motions, and it can come up with some arguments that you might not have thought of. You just have to ignore all its citations and look everything up yourself, otherwise this:
https://www.reuters.com/legal/new-york-lawyers-sanctioned-us...
Re: ChatGPT-4 significantly increased performance of business consultants
#146Earlier quoted context omitted.
I like Cory Doctorow’s use in “Chickenized Reverse Centaur”.
Powerful stuff. If anyone wants a short explanation here's one: https://m.youtube.com/watch?v=6pieIoEi8Ds I love it when a complicated set of conditions can be losslessly encoded in a short, easily remembered label.
This was really profound, thank you.
Re: ChatGPT-4 significantly increased performance of business consultants
#147So useless BCG consultants were faster in delivery bullshit with ChatGPT? That's impressive.
Re: ChatGPT-4 significantly increased performance of business consultants
#148Well, they buried the lede with this one. Using LLMs were better for some tasks and actually made it worse for others. The first task was a generalist task ("inside the frontier" as they refer to it), which I'm not surprised has improved performance, as it purposely made to fall into an LLM's areas of strength: research into well-defined areas where you might not have strong domain knowledge. This also is the mainsta…
I was thinking about this last night. It’s a new version of Gell-Mann amnesia. I call it LLm-man amnesia. When I ask a programming question, chat GPT hallucinates something about 20% of the time and I can only tell because I’m skilled enough to see it. For all the other domains I ask it questions if I should assume at least as much hallucination and incorrect information.
I see this as for drill-down thinking from a broad -> specific concept AI seems to be helpful when supplementing specialist work. However like you both mentioned: when needing more focused and integrated answers AI tends hinders performance.
However as the paper noted, when working within AIs areas of strength it improved not only efficiency but the quality of the work as well (accounting for the hallucinations). As you mentioned:
> When I ask a programming question, chat GPT hallucinates something about 20% of the time and I can only tell because I’m skilled enough to see it
This matches their Centaur approach, delineating between AI and one’s own skills for a task which—with generalized work—seems to fair better than not using AI at all.
Re: ChatGPT-4 significantly increased performance of business consultants
#149Earlier quoted context omitted.
>LLMs are not worthless for programming. They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards. For an example. I've used Gpt4 as a sort of Google on steroids with prompts like "do subnets in gcloud span azs" and ", "in gcloud secret manager can you access secrets across regions". I very quickly learned to ask "is it true" after every answer…
> They can be worse than worthless. They can sabotage your work if you let them making you spend even more time fixing it afterwards. I basically gave up on llms because i was spending more time figuring out what it did wrong than actually getting value. People without programming skill are still impressed by them. But they yet have to learn or deliver anything of value even with the help of chat bots.
I have twenty years of programming experience and LLMs give me a significant productivity boost for programming on a daily basis: https://simonwillison.net/2023/Sep/29/llms-podcast/#code-int...
Re: ChatGPT-4 significantly increased performance of business consultants
#150Well, they buried the lede with this one. Using LLMs were better for some tasks and actually made it worse for others. The first task was a generalist task ("inside the frontier" as they refer to it), which I'm not surprised has improved performance, as it purposely made to fall into an LLM's areas of strength: research into well-defined areas where you might not have strong domain knowledge. This also is the mainsta…
LLMs are broadly good at things that average knowledge workers are good at or can be trained to be good at reasonably quickly.