Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT agent: bridging research and action

openai.com

Post reply on HN
Print view View original

441–450 of 508 posts

Re: ChatGPT agent: bridging research and action

#441

Post by baxtr »

satvikpendem wrote: post #94
ricardobayes wrote: post #76

Earlier quoted context omitted.

Of course, Pareto principle is at work here. In an adjacent field, self-driving, they are working on the last "20%" for almost a decade now. It feels kind of odd that almost no one is talking about self-driving now, compared to how hot of a topic it used to be, with a lot of deep, moral, almost philosophical discussions.

> The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time. — Tom Cargill, Bell Labs https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule

Love this quote. Kinda sad I didn’t see this earlier in my life.

Re: ChatGPT agent: bridging research and action

#442

Post by ashwindharne »

lairv wrote: post #213
AgentMatrixAI wrote: post #47

I'm not so optimistic as someone that works on agents for businesses and creating tools for it. The leap from low 90s to 99% is classic last mile problem for LLM agents. The more generic and spread an agent is (can-do-it-all) the more likely it will fail and disappoint. Can't help but feel many are optimizing happy paths in their demos and hiding the true reality. Doesn't mean there isn't a place for agents but rathe…

In general most of the previous AI "breakthrough" in the last decade were backed by proper scientific research and ideas: - AlphaGo/AlphaZero (MCTS) - OpenAI Five (PPO) - GPT 1/2/3 (Transformers) - Dall-e 1/2, Stable Diffusion (CLIP, Diffusion) - ChatGPT (RLHF) - SORA (Diffusion Transformers) "Agents" is a marketing term and isn't backed by anything. There is little data available, so it's hard to have generally capa…

'Agents' are just a design pattern for applications that leverage recent proper scientific breakthroughs. We now have models that are increasingly capable of reading arbitrary text and outputting valid json/xml. It seems like if we're careful about what text we feed them and what json/xml we ask for, we can get them to string together meaningful workflows and operations.

Obviously, this is working better in some problem spaces than others; seems to mainly depend on how in-distribution the data domain is to the LLM's training set. Choices about context selection and the API surface exposed in function calls seem to have a large effect on how well these models can do useful work as well.

Re: ChatGPT agent: bridging research and action

#443

Post by NoboruWataya »

twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

Also... he "thinks" it got 98% of the data correct. How does he know?

Re: ChatGPT agent: bridging research and action

#444

Post by schmidtleonard »

intended wrote: post #376
pbh101 wrote: post #311

Earlier quoted context omitted.

This is a big moving of the goalposts. The optimists were saying Level 5 would be purchasable everywhere by ~2018. They aren’t purchasable today, just hail-able. And there’s a lot of remote human intervention. And San Francisco doesn’t get snow.

Hell - SF doesn’t have motorcyclists or any vehicular traffic, driving on the wrong side of the road. Or cows sharing the thoroughfares. It should be obvious to all HNers that have lived or travelled to developing / global south regions - driving data is cultural data. You may as well say that self driving will only happen in countries where the local norms and driving culture is suitable to the task. A desperately a…

Yes, but they are getting good at chasing 9s in the US, those skills will translate directly to chasing 9s outside the US, and frankly the "first drafts" did quite a bit better than I'd have expected even six months ago

Guangzhou: https://www.youtube.com/watch?v=3DWz1TD-VZg

Paris: https://www.youtube.com/watch?v=iN9nu-IkS1w

Rome: https://www.youtube.com/watch?v=4Zg3jc90JTI

Re: ChatGPT agent: bridging research and action

#445

Post by vonneumannstan »

twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

>It feels like either finding that 2% that's off (or dealing with 2% error) will be the time consuming part in a lot of cases.

People act like this is some new thing but this exactly what supervising a more junior coworker is like. These models won't stay performing at Jr. levels for long. That is clear

Re: ChatGPT agent: bridging research and action

#446

Post by LgLasagnaModel »

vonneumannstan wrote: post #445
twalkz wrote: post #30

The "spreadsheet" example video is kind of funny: guy talks about how it normally takes him 4 to 8 hours to put together complicated, data-heavy reports. Now he fires off an agent request, goes to walk his dog, and comes back to a downloadable spreadsheet of dense data, which he pulls up and says "I think it got 98% of the information correct... I just needed to copy / paste a few things. If it can do 90 - 95% of the…

>It feels like either finding that 2% that's off (or dealing with 2% error) will be the time consuming part in a lot of cases. People act like this is some new thing but this exactly what supervising a more junior coworker is like. These models won't stay performing at Jr. levels for long. That is clear

It is not exactly like that. Junior coworkers mess up in fairly predictable ways.

Re: ChatGPT agent: bridging research and action

#447

Post by jazzyjackson »

idiotsecant wrote: post #438
psychoslave wrote: post #397

Earlier quoted context omitted.

What a awful way to think about internship. The goal is to help people grow, so they can achieve things they would not have been able to deal with before gaining that additional experience. This might include boring dirty work, yes. But that means they thus prove they can overcome such a struggle, and so more experienced people should be expected to also be able to go though it - if there is no obvious more pleasant…

The goal of internships in a for profit company is not the personal growth of the intern. This is a nice sentiment but the function of the company is to make money, so an intern with net negative productivity doesn't make sense when goals are quarterly financials.

Sure, companies wouldn't do anything that negatively affects their bottom line, but consider the case that an intern is a net zero - they do some free labor equal to the drag they cause demanding attention of their mentor. Why have an intern in that case? Because long term, expanding the talent pool suppresses wages. Increasing the number of qualified candidates gives power to the employer. The "Learn to Code" campaign along with the litany of code bootcamps is a great example, it poses as personal growth / job training to increase the earning power of individuals, but on the other side of that is an industry that doesn't want to pay its workers 6 figures, so they want to make coding a blue collar job.

But coding didn't become a low wage job, now we're spending GPU credits to make pull requests instead and skipping the labor all together. Anyway I share the parent poster's chagrin at all the comparisons of AI to an intern. If all of your attention is spent correcting the work of a GPU, the next generation of workers will never have mentors giving them attention, starving off the supply of experienced entry level employees. So what happens in 10, 20 years ? I guess anyone who actually knows how to debug computers instead of handing the problem off to an LLM will command extraordinary emergency-fix-it wages.

Re: ChatGPT agent: bridging research and action

#448

Post by addandsubtract »

j_timberlake wrote: post #377
dcre wrote: post #65

Very slightly impressed by their emphasis on the gigantic (my word, not theirs) risk of giving the thing access to real creds and sensitive info.

The sane way to do this (if you wanted to) would be to give the AI a debit card with a small balance to work with. If funds get stolen, you know exactly what the maximum damage is. And if you can't afford that damage, then you wouldn't have been able to afford that card to begin with. But since people can cancel transactions with a credit card, that's what people are going to do, and it will be a huge mess every time…

Most online merchants have you verify a payment on your phone, too. Maybe you can even activate it with your bank to ask your every time.

Re: ChatGPT agent: bridging research and action

#449

Post by wat10000 »

stockresearcher wrote: post #312
simantel wrote: post #197

Earlier quoted context omitted.

> It feels kind of odd that almost no one is talking about self-driving now, compared to how hot of a topic it used to be Probably because it's just here now? More people take Waymo than Lyft each day in SF.

The typical Lyft vehicle is a piece of junk worth less than $20k, while the typical Waymo vehicle is a pretend luxury car with $$$ of equipment tacked on. Waymo needs to be proving 5-10x the number of daily rides as Lyft before we get excited

I suspect most gig drivers don't fully account for the cost of running their car, so these services are also being effectively subsidized by their workers.

You can provide almost any service at a loss, for a while, with enough money. We shouldn't get excited until Waymo starts turning an actual profit.

Re: ChatGPT agent: bridging research and action

#450

Post by wat10000 »

n2d4 wrote: post #351
amccollum wrote: post #240

Earlier quoted context omitted.

People were saying we would all be getting in our cars and taking a nap on our morning commute. We are clearly still a pretty long ways off from self-driving being as ubiquitous as it was claimed it would be.

There are always extremists with absurd timelines on any topic! (Didn't people think we'd be on Mars in 2020?) But this one? In the right cities, plenty of people take a Waymo morning commute every day. I'd say self-driving cars have been pretty successful at meeting people's expectations — or maybe you and I are thinking of different people.

The expectation of a "self-driving car" is that you can get in it and take any trip that a human driver could take. The "in certain cities" is a huge caveat. If we accept that sort of geographical limitation, why not say that self-driving "cars" have been a thing since driverless metro systems started showing up in the 1980s?
Post reply on HN