Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT agent: bridging research and action

openai.com

Post reply on HN
Print view View original

111–120 of 508 posts

Re: ChatGPT agent: bridging research and action

#111

Post by torginus »

samtp wrote: post #82
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

This is the exact same issue that I've had trying to use LLMs for anything that needs to be precise such as multi-step data pipelines. The code it produces will look correct and produce a result that seems correct. But when you do quality checks on the end data, you'll notice that things are not adding up. So then you have to dig into all this overly verbose code to identify the 3-4 subtle flaws with how it transform…

I'll get into hot water with this, but I still think LLMs do not think like humans do - as in the code is not a result of a trying to recreate a correct thought process in a programming language, but some sort of statistically most likely string that matches the input requirements,

I used to have a non-technical manager like this - he'd watch out for the words I (and other engineers) said and in what context, and would repeat them back mostly in accurate word contexts. He sounded remarkably like he knew what he was talking about, but would occasionally make a baffling mistake - like mixing up CDN and CSS.

LLMs are like this, I often see Cursor with Claude making the same kind of strange mistake, only to catch itself in the act, and fix the code (but what happens when it doesn't)

Re: ChatGPT agent: bridging research and action

#112

Post by exitb »

FergusArgyll wrote: post #103
pants2 wrote: post #70

I've been using OpenAI operator for some time - but more and more websites are blocking it, such as LinkedIn and Amazon. That's two key use-cases gone (applying to jobs and online shopping). Operator is pretty low-key, but once Agent starts getting popular, more sites will block it. They'll need to allow a proxy configuration or something like that.

If people will actually pay for stuff (food, clothing, flights, whatever) through this agent or operator, I see no reason Amazon etc would continue to block them.

Many shopping experiences are oriented towards selling you more than you originally wanted to buy. This doesn’t work if a robot is doing the buying.

Re: ChatGPT agent: bridging research and action

#113

Post by torginus »

pants2 wrote: post #70

I've been using OpenAI operator for some time - but more and more websites are blocking it, such as LinkedIn and Amazon. That's two key use-cases gone (applying to jobs and online shopping). Operator is pretty low-key, but once Agent starts getting popular, more sites will block it. They'll need to allow a proxy configuration or something like that.

Maybe it'll red team reason a scraper into existence :)

Re: ChatGPT agent: bridging research and action

#114

Post by ishita159 »

dingnuts wrote: post #106
ricardobayes wrote: post #76

Earlier quoted context omitted.

Of course, Pareto principle is at work here. In an adjacent field, self-driving, they are working on the last "20%" for almost a decade now. It feels kind of odd that almost no one is talking about self-driving now, compared to how hot of a topic it used to be, with a lot of deep, moral, almost philosophical discussions.

The critics of the current AI buzz certainly have been drawing comparisons to self driving cars as LLMs inch along with their logarithmic curve of improvement that's been clear since the GPT-2 days. Whenever someone tells me how these models are going to make white collar professions obsolete in five years, I remind them that the people making these predictions 1) said we'd have self driving cars "in a few years" bac…

I think people don't realize how much models have to extrapolate still, which causes hallucinations. We are still not great at giving all the context in our brain to LLMs.

There's still a lot of tooling to be built before it can start completely replacing anyone.

Re: ChatGPT agent: bridging research and action

#115

Post by pants2 »

FergusArgyll wrote: post #103
pants2 wrote: post #70

I've been using OpenAI operator for some time - but more and more websites are blocking it, such as LinkedIn and Amazon. That's two key use-cases gone (applying to jobs and online shopping). Operator is pretty low-key, but once Agent starts getting popular, more sites will block it. They'll need to allow a proxy configuration or something like that.

If people will actually pay for stuff (food, clothing, flights, whatever) through this agent or operator, I see no reason Amazon etc would continue to block them.

I was buying plenty of stuff through Amazon before they blocked Operator. Now I sometimes buy through other sites that allow it.

The most useful for me was: "here's a picture of a thing I need a new one of, find the best deal and order it for me. Check coupon websites to make sure any relevant discounts are applied."

To be honest, if Amazon continues to block "Agent Mode" and Walmart or another competitor allows it, I will be canceling Prime and moving to that competitor.

Re: ChatGPT agent: bridging research and action

#116

Post by doctorpangloss »

dingnuts wrote: post #106
ricardobayes wrote: post #76

Earlier quoted context omitted.

Of course, Pareto principle is at work here. In an adjacent field, self-driving, they are working on the last "20%" for almost a decade now. It feels kind of odd that almost no one is talking about self-driving now, compared to how hot of a topic it used to be, with a lot of deep, moral, almost philosophical discussions.

The critics of the current AI buzz certainly have been drawing comparisons to self driving cars as LLMs inch along with their logarithmic curve of improvement that's been clear since the GPT-2 days. Whenever someone tells me how these models are going to make white collar professions obsolete in five years, I remind them that the people making these predictions 1) said we'd have self driving cars "in a few years" bac…

Okay, but the experts saying self driving cars were 50 years out in 2015 were wrong too. Lots of people were there for those speeches, and yet, even the most cynical take on Waymo, Cruise and Zoox’s limitations would concede that the vehicles are autonomous most of the time in a technologically important way.

There’s more to this than “predictions are hard.” There are very powerful incentives to eliminate driving and bloated administrative workforces. This is why we don’t have flying cars: lack of demand. But for “not driving?” Nobody wants to drive!

Re: ChatGPT agent: bridging research and action

#117

Post by esafak »

causalmodels wrote: post #96
esafak wrote: post #86

Earlier quoted context omitted.

There needs to be a profit sharing scheme. This is the same reason publishers didn't like Google providing answers instead of links.

Why does an ecommerce website need a profit sharing agreement?

Why would they want an LLM to slurp their web site to help some analyst create a report about the cost of widgets? If they value the data they can pay for it. If not, they don't need to slurp it, right? This goes for training data too.

Re: ChatGPT agent: bridging research and action

#118

Post by pants2 »

jorisboris wrote: post #87
pants2 wrote: post #70

I've been using OpenAI operator for some time - but more and more websites are blocking it, such as LinkedIn and Amazon. That's two key use-cases gone (applying to jobs and online shopping). Operator is pretty low-key, but once Agent starts getting popular, more sites will block it. They'll need to allow a proxy configuration or something like that.

How do they block it?

Certainly there's a fixed IP range or browser agent that OpenAI uses

Re: ChatGPT agent: bridging research and action

#119

Post by pants2 »

atmosx wrote: post #90
pants2 wrote: post #70

I've been using OpenAI operator for some time - but more and more websites are blocking it, such as LinkedIn and Amazon. That's two key use-cases gone (applying to jobs and online shopping). Operator is pretty low-key, but once Agent starts getting popular, more sites will block it. They'll need to allow a proxy configuration or something like that.

There are companies that sell the entire dataset of these websites :-) - it’s just one phone call away to solve on the OpenAI side.

It's not about the data, it's about "operating" the site to buy things for you.

Re: ChatGPT agent: bridging research and action

#120

Post by joshstrange »

travelalberta wrote: post #59
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

I think this is my favorite part of the LLM hype train: the butterfly effect of dependence on an undependable stochastic system propagates errors up the chain until the whole system is worthless. "I think it got 98% of the information correct..." how do you know how much is correct without doing the whole thing properly yourself? The two options are: - Do the whole thing yourself to validate - Skim 40% of it, 'seems…

> I think the funny part is that humans are not exempt from similar mistakes, but a human making those mistakes again and again would get fired. Meanwhile an agent that you accept to get only 98% of things right is meeting expectations.

My rule is that if you submit code/whatever and it has problems you are responsible for them no matter how you "wrote" it. Put another way "The LLM made a mistake" is not a valid excuse nor is "That's what the LLM spit out" a valid response to "why did you write this code this way?".

LLMs are tools, tools used by humans. The human kicking off an agent, or rather submitting the final work, is still on the hook for what they submit.

Post reply on HN