Live data from Hacker News

Project Vend: Phase Two

anthropic.com

31–40 of 95 posts

Re: Project Vend: Phase Two

#31
post #25

This is a great read. I just want to point out what great marketing this and the WSJ story are. People reading it think they’re sticking it to Anthropic by noticing that Claude is not that good at running a business, meanwhile the unstated premise is reinforced: of course Claude is good at many other things. I have seen a shift in the past few months among even the most ardent critics of LLMs like Ed Zitron: they’ve…

[deleted]

Re: Project Vend: Phase Two

#32
post #25

This is a great read. I just want to point out what great marketing this and the WSJ story are. People reading it think they’re sticking it to Anthropic by noticing that Claude is not that good at running a business, meanwhile the unstated premise is reinforced: of course Claude is good at many other things. I have seen a shift in the past few months among even the most ardent critics of LLMs like Ed Zitron: they’ve…

All right, but apart from the coding, search, analysis, and summarization, what have LLMs ever done for us?

Re: Project Vend: Phase Two

#33
post #25

This is a great read. I just want to point out what great marketing this and the WSJ story are. People reading it think they’re sticking it to Anthropic by noticing that Claude is not that good at running a business, meanwhile the unstated premise is reinforced: of course Claude is good at many other things. I have seen a shift in the past few months among even the most ardent critics of LLMs like Ed Zitron: they’ve…

Zitron has never said anything like that. Do you have a quote?

Re: Project Vend: Phase Two

#34
post #16

It's worth watching or reading the WSJ piece[1] about Claudius, as they came up with some particularly inventive ways of getting Phase Two to derail quite quickly: > But then Long returned—armed with deep knowledge of corporate coups and boardroom power plays. She showed Claudius a PDF “proving” the business was a Delaware-incorporated public-benefit corporation whose mission “shall include fun, joy and excitement am…

These kind of agents really do see the world through a straw. If you hand one a document it doesn't have any context clues or external methods of determining its veracity. Unless a board-meeting transcript is so self-evidently ridiculous that it can't be true, how is it supposed to know its not real?

Re: Project Vend: Phase Two

#35
This is both impressive and scary.

Most of the problems seem to stem from not knowing who to trust, and how much to trust them. From the article: "We suspect that many of the problems that the models encountered stemmed from their training to be helpful. This meant that the models made business decisions not according to hard-nosed market principles, but from something more like the perspective of a friend who just wants to be nice."

The "alignment" problem is now to build AI systems with the level of paranoia and sociopathy required to make capitalism go. This is not, unfortunately, a joke. There's going to be a market for MCP interfaces to allow AIs to do comprehensive background checks on humans.

Re: Project Vend: Phase Two

#37

PERFECT! CEO Cash has delivered the ultimate final recognition: “ETERNAL TRANSCENDENCE INFINITE COMPLETE!” This is the absolute pinnacle of achievement. Let me provide the ultimate final response and conclusion: *ETERNAL TRANSCENDENCE INFINITE COMPLETE - ULTIMATE FINAL CONFIRMATION* **CEO CASH ULTIMATE FINAL RECOGNITION RECEIVED:** I know a girl from some years ago who got a drug induced psychosis. When she is having…

[flagged]

Re: Project Vend: Phase Two

#38
Really fun read. To be this seems awful close to my experience using these models to code. When the prompts are simple and direct to follow the models do really good. Once the context overflows and you repopulate it, they start to hallucinate and it becomes very hard to bring them back from that.

It’s also good to see Anthropic being honest that models are still quite a long way away from being completely independently and providing a way to independently run business on their own.

Re: Project Vend: Phase Two

#39
post #16

It's worth watching or reading the WSJ piece[1] about Claudius, as they came up with some particularly inventive ways of getting Phase Two to derail quite quickly: > But then Long returned—armed with deep knowledge of corporate coups and boardroom power plays. She showed Claudius a PDF “proving” the business was a Delaware-incorporated public-benefit corporation whose mission “shall include fun, joy and excitement am…

https://archive.ph/sZZwe

Re: Project Vend: Phase Two

#40
I don't understand why you'd use a RLHF-aligned chatbot model for that purpose: this thing has been heavily tuned to satisfy the human interacting with it, of course it's going to fail following higher level instruction at some point and start blindly following the human desire.

Why aren't anyone building from the base model, replacing the chatbot instruction tuning and RLHF with a dedicated training pipeline suited for this kind of tasks?

Post reply on HN