Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT agent: bridging research and action

openai.com

Post reply on HN
Print view View original

481–490 of 508 posts

Re: ChatGPT agent: bridging research and action

#481

Post by ytpete »

j_timberlake wrote: post #364
2oMg3YWV26eKIs wrote: post #167

The security risks with this sound scary. Let's say you give it access to your email and calendar. Now it knows all of your deepest secrets. The linked article acknowledges that prompt injection is a risk for the agent: > Prompt injections are attempts by third parties to manipulate its behavior through malicious instructions that ChatGPT agent may encounter on the web while completing a task. For example, a maliciou…

Create a burner account for email/calendar, that solves most of those problems. Nobody will care if the AI leaks that you have a dentist appointment on Tuesday.

But isn't the whole supposed value-add here that it gets access to your real data? If you don't want it to get at your calendar, you could presumably just not grant it access in the first place – no need for a fake one. But if you want it to automatically "book me a haircut with the same person as last time in an afternoon time slot when I'm free later this month" then it needs access to your real calendar and if attacked it can leak or wreck your real calendar too. It's hard to see how you can ever have one without the other.

Re: ChatGPT agent: bridging research and action

#482

Post by ytpete »

camdenreslink wrote: post #459
posix86 wrote: post #400

Earlier quoted context omitted.

But that's how progress works! To me it makes sense that llms first manage to do 80% of the task, then 90, then 95, then 98, then 99, then 99.5, and so on. The last part IS the hardest, and each iteration of LLMs will get a bit further. Just because it didn't reach 100% just yet doesn't mean that LLMs as a whole are doomed. In fact, the fact that they are slowly approaching 100% shows promise that there IS a future f…

But they don’t do 80% of the task. They do 100% of the task, but 20% is wrong (and you don’t know which 20% without manually verifying all of it). So it is really great for tasks where do the work is a lot harder than verifying it, and mostly useless for tasks where doing the work and verifying it are similarly difficult.

Right — and I'd conjecture until LLMs get close to the accuracy of an entry-level employee, they may not have enough economic value to be viable beyond the hype/novelty phase. Why? Because companies already chose a "minimum quality to be valuable" bar when they set the bar for their most junior entry level. They could get lower-quality work for cheaper by just carving out an even lower-bar hiring tier. If they haven't, maybe it's because work below that quality level is just not a net-positive contribution at all.

Re: ChatGPT agent: bridging research and action

#483

Post by intended »

schmidtleonard wrote: post #444
intended wrote: post #376

Earlier quoted context omitted.

Hell - SF doesn’t have motorcyclists or any vehicular traffic, driving on the wrong side of the road. Or cows sharing the thoroughfares. It should be obvious to all HNers that have lived or travelled to developing / global south regions - driving data is cultural data. You may as well say that self driving will only happen in countries where the local norms and driving culture is suitable to the task. A desperately a…

Yes, but they are getting good at chasing 9s in the US, those skills will translate directly to chasing 9s outside the US, and frankly the "first drafts" did quite a bit better than I'd have expected even six months ago Guangzhou: https://www.youtube.com/watch?v=3DWz1TD-VZg Paris: https://www.youtube.com/watch?v=iN9nu-IkS1w Rome: https://www.youtube.com/watch?v=4Zg3jc90JTI

I’m rejecting the assertion that the data covers a physics model - which would be invariant across nations.

I’m positing that the models encode cultural decision making norms- and using global south regions to highlight examples of cases that are commonplace but challenge the feasibility of full autonomous driving.

Imagine an auto rickshaw with full self driving.

If in your imagination, you can see a level 5 auto, jousting for position in Mumbai traffic - then you have an image which works.

It’s also well beyond what people expect fully autonomous driving entails.

At that point you are encoding cultural norms and expectations around rule/law enforcement.

Re: ChatGPT agent: bridging research and action

#484

Post by intended »

wyclif wrote: post #381
intended wrote: post #376

Earlier quoted context omitted.

Hell - SF doesn’t have motorcyclists or any vehicular traffic, driving on the wrong side of the road. Or cows sharing the thoroughfares. It should be obvious to all HNers that have lived or travelled to developing / global south regions - driving data is cultural data. You may as well say that self driving will only happen in countries where the local norms and driving culture is suitable to the task. A desperately a…

I'm in the Philippines now, and that's how I know this is the correct take. Especially this part: "Driving data is cultural data." The optimists underestimate a lot of things about self-driving cars. The biggest one may be that in developing and global south regions, civil engineering, design, and planning are far, far away from being up to snuff to a level where Level 5 is even a slim possibility. Here on the island…

Level 5 is a pipe dream. Or if I’m being charitable it’s un-ambitious.

The goal for a working L5 should be “if piloting a rickshaw, will it be able to operate as a human owner in normal traffic.”

Re: ChatGPT agent: bridging research and action

#485

Post by danielmarkbruce »

psychoslave wrote: post #397
slg wrote: post #225

Earlier quoted context omitted.

The proper use of these systems is to treat them like an intern or new grad hire. You can give them the work that none of the mid-tier or senior people want to do, thereby speeding up the team. But you will have to review their work thoroughly because there is a good chance they have no idea what they are actually doing. If you give them mission-critical work that demands accuracy or just let them have free rein with…

What a awful way to think about internship. The goal is to help people grow, so they can achieve things they would not have been able to deal with before gaining that additional experience. This might include boring dirty work, yes. But that means they thus prove they can overcome such a struggle, and so more experienced people should be expected to also be able to go though it - if there is no obvious more pleasant…

What an awful way to think about other people, always assuming the very worst version of what they said.

Re: ChatGPT agent: bridging research and action

#486

Post by j_timberlake »

camdenreslink wrote: post #455
j_timberlake wrote: post #359

Earlier quoted context omitted.

"It feels like either finding that 2% that's off (or dealing with 2% error) will be the time consuming part in a lot of cases." This is the part you have wrong. People just won't do that. They'll save the 8 hours and just deal with 2% error in their work (which reduces as AI models get better). This doesn't work with something with a low error tolerance, but most people aren't building the next Golden Gate Bridge. Th…

It’s not worth it because of the compounding effect when it is a repeated process. 98% accuracy might be fine for a single iteration, but if you run your process 365 times (maybe once a day for a year) whatever your output is will be so wrong that it is unusable.

Can you name a single job like this? It's much easier to name jobs where the accuracy doesn't compound, like daily customer service chatbots, or personal-tutor bots, or news-aggregator bots, or the inevitable (and somewhat dubious) do-my-tax-returns bot.

All I can think of is vibe-coding, and vibe-coding jobs aren't a thing.

Re: ChatGPT agent: bridging research and action

#487

Post by econ »

j_timberlake wrote: post #486
camdenreslink wrote: post #455

Earlier quoted context omitted.

It’s not worth it because of the compounding effect when it is a repeated process. 98% accuracy might be fine for a single iteration, but if you run your process 365 times (maybe once a day for a year) whatever your output is will be so wrong that it is unusable.

Can you name a single job like this? It's much easier to name jobs where the accuracy doesn't compound, like daily customer service chatbots, or personal-tutor bots, or news-aggregator bots, or the inevitable (and somewhat dubious) do-my-tax-returns bot. All I can think of is vibe-coding, and vibe-coding jobs aren't a thing.

Doctors get the diagnosis wrong 10-23% of the time (depending on who you ask)

Re: ChatGPT agent: bridging research and action

#488

Post by schmidtleonard »

intended wrote: post #483
schmidtleonard wrote: post #444

Earlier quoted context omitted.

Yes, but they are getting good at chasing 9s in the US, those skills will translate directly to chasing 9s outside the US, and frankly the "first drafts" did quite a bit better than I'd have expected even six months ago Guangzhou: https://www.youtube.com/watch?v=3DWz1TD-VZg Paris: https://www.youtube.com/watch?v=iN9nu-IkS1w Rome: https://www.youtube.com/watch?v=4Zg3jc90JTI

I’m rejecting the assertion that the data covers a physics model - which would be invariant across nations. I’m positing that the models encode cultural decision making norms- and using global south regions to highlight examples of cases that are commonplace but challenge the feasibility of full autonomous driving. Imagine an auto rickshaw with full self driving. If in your imagination, you can see a level 5 auto, jo…

You're not wrong on the "physics easy culture hard" call, just late. That was Andrej Karpathy's stated reason for betting on the Tesla approach over the Waymo approach back in 2017, because he identified that the limiting factor would be the collection of data on real-world driving interactions in diverse environments to allow learning theories-of-mind for all actors across all settings and cultures. Putting cameras on millions of cars in every corner of the world was the way to win that game -- simulations wouldn't cut it, "NPC behavior" would be their downfall.

This bet aged well: videos of FSD performing very well in wildly different settings -- crowded Guangzhou markets to French traffic circles to left-hand-drive countries -- seem to indicate that this approach is working. It's nailing interactions that it didn't learn from suburban America and that require inferring intent using complex contextual clues. It's not done until it's done, but the god of the gaps retreats ever further into the march of nines and you don't get credit for predicting something once it has already happened.

Re: ChatGPT agent: bridging research and action

#489

Post by bluefirebrand »

coderatlarge wrote: post #437
bluefirebrand wrote: post #320

Earlier quoted context omitted.

The point of coaching a Junior is so they improve their skills for next time What would be the point of coaching an LLM? You will just have to coach it again and again

coaching a junior doesn’t just improve the junior. It also tends to improve the senior.

Coaching an LLM seems unlikely to improve you meaningfully

Re: ChatGPT agent: bridging research and action

#490

Post by wat10000 »

vidarh wrote: post #480
wat10000 wrote: post #452

Earlier quoted context omitted.

A token stream is universal, but I don't see any reason to think that a token stream generated by an LLM can ever be universal. I mean, theoretically in an "infinite tape" model, sure. But we don't even know if it's physically possible. Given that the observable universe is finite and the information capacity of a finite space is also finite, then anything humans can do can theoretically be encoded with a lookup tabl…

Unless humans exceed the Turing computable, the human brain is the existence proof that a sufficiently complex Turing machine can be made to replicate human thought in a compact space. That encoding a naive/basic UTM in an LLM would potentially be impractical is largely irrelevant in that case, because for any UTM you can "compress" the program by increasing the number of states or symbols, and effectively "embedding…

You seem to be making a giant leap from “human thought can probably be emulated by a Turing machine” to “human thought can probably be emulated by LLMs in the actual physical universe.” The former is obvious, the latter I’m deeply skeptical of.

The machine part of a Turing machine is simple. People manage to build them by accident. Programming language designers come up with a nice-sounding type inference feature and discover that they’ve made their type system Turing-complete. The hard part is the execution speed and the infinite tape.

Ignoring those problems, making AGI with LLMs is easy. You don’t even need something that big. Make a neural network big enough to represent the transition table of a Turing machine with a dozen or so states. Configure it to be a universal machine. Then give it a tape containing a program that emulates the known laws of physics to arbitrary accuracy. Simulate the universe from the Big Bang and find the people who show up about 13 billion years later. If the known laws of physics aren’t accurate enough, compare with real-world data and adjust as needed.

There’s the minor detail that simulating quantum mechanics takes time exponential in the number of particles, and the information needed to represent the entire universe can’t fit into that same universe and still leave room for anything else, but that doesn’t matter when you’re talking Turing machines.

It does matter a great deal when talking about what might lead to actual human-level intelligent machines existing in reality, though.

Post reply on HN