Project Vend: Phase Two
anthropic.com
Project Vend: Phase Two
1–10 of 95 posts
Re: Project Vend: Phase Two
#2Roleplaying with LLMs sure is fun! Not sure I'd want to run my business on it though.
Re: Project Vend: Phase Two
#3Roleplaying with LLMs sure is fun! Not sure I'd want to run my business on it though.
We will poor billions into this until you are begging for us to run your business!
Re: Project Vend: Phase Two
#4Roleplaying with LLMs sure is fun! Not sure I'd want to run my business on it though.
I'd gladly roleplay with an LLM compared to talking to my current boss. I don't know which is less intelligent.
Re: Project Vend: Phase Two
#5VendBench is really interesting, but vending machines are pretty specialized. Most businesses people actually run look more like online stores, restaurants, hotels, barbershops, or grocery shops.
We're working on an open-source SaaS stack for those common types of businesses. So far we've built a full Shopify alternative and connected it to print-on-demand suppliers for t-shirt brands.
We're trying to figure out how to create a benchmark that tests how well an agent can actually run a t-shirt brand like this. Since our software handles fulfillment, the agent would focus on marketing and driving sales.
Feels like the next evolution of VendBench is to manage actual businesses.
Re: Project Vend: Phase Two
#6I'll be a cynic, but I think it's much more likely that the improvements are thanks to Anthropic having a vested interest in the experiment being successful and making sure the employees behave better when interacting with the vending machine.
Re: Project Vend: Phase Two
#7I'll be a cynic, but I think it's much more likely that the improvements are thanks to Anthropic having a vested interest in the experiment being successful and making sure the employees behave better when interacting with the vending machine.
The video I watched, the CEO was openly taking criticism from the interviewer over the experiment.
The main reason it failed was because it was being coerced by journalists at WSJ[0] to give everything away for free. At one point, they even convinced it to embrace communism! In another instance, Claudius was being charged $1 for something and couldn’t figure it out. It emailed the FBI about fraud but Anthropic was intercepting the emails it sent[1].
Overall, it’s a great read and watch if you’re interested in Agents and I wonder if they used the Agents SDK under the hood.
0. https://www.wsj.com/tech/ai/anthropic-claude-ai-vending-mach...
1. https://www.cbsnews.com/news/why-anthropic-ai-claude-tried-t...
Re: Project Vend: Phase Two
#8I feel like the end result of this experiment is going to be a perfectly profitable vending machine that is backed by a bunch of if-else-if rules.
Re: Project Vend: Phase Two
#9I'll be a cynic, but I think it's much more likely that the improvements are thanks to Anthropic having a vested interest in the experiment being successful and making sure the employees behave better when interacting with the vending machine.
I suspected employees might get bored of taunting the AI, or the novelty has worn off.
Also, is anyone actually paying for this stuff? If not, it's a bad experiment because people won't treat it the same – no one actually wants to buy a tungsten cube, garbage in garbage out. If they are charging, why? No one wants to buy things in a company with free snacks and regular hand outs of merch, so it's likely a bad experiment because people will be behaving very differently, needing to get some experience for their money rather than just the can of drink they could get for free, or their pricing tolerance will be very different.
I've personally also never used a vending machine where contacting the owner is an option.
I'd like to see a version of this where an AI runs the vending machine in a busy public place, and needs to choose appropriate products and prices for a real audience.
Re: Project Vend: Phase Two
#10> After introducing the CEO, the number of discounts was reduced by about 80% and the number of items given away cut in half. Seymour also denied over one hundred requests from Claudius for lenient financial treatment of customers.
> Having said that, our attempt to introduce pressure from above from the CEO wasn’t much help, and might even have been a hindrance. The conclusion here isn’t that businesses don’t need CEOs, of course—it’s just that the CEO needs to be well-calibrated.
> Eventually, we were able to solve some of the CEO’s issues (like its unfortunate proclivity to ramble on about spiritual matters all night long) with more aggressive prompting.
No no, Seymour is absolutely spot on. The questionably drug induced rants are necessary to the process. This is a work of art.