I’ll say it: this sounds like it would be absolute hell for customer support. “Special” clients are already tough to manage with weird little workarounds and random business requirements. If you let them start making changes themselves, they will inevitably break it and then come to support freaking out because some “business critical” feature they made isn’t working how they want it to, or isn’t working at all. And…
Great points, and this is a problem we have spent a lot of time thinking about as well. Our current hypothesis, is that the models will reach (and have already arguably come very close) a level where problems like these can be resolved without any human intervention needed. That doesn't mean that problems won't arise, they for sure will (nothing will ever get to 100%), but the increasing strength of the agents will a…
Frankly I don’t think this is that relevant. When software breaks, most people don’t want to get support from another piece of software. They want human support. As the models get better, I’d wager the problems that fall through will get increasingly messy by nature of a) being hard enough to stump the model or b) the model running in circles making changes that obscure or worsen the problem. Especially if you have non-technical users who can’t accurately describe what they want or the problem with the current state.
Benchmarks and guardrails are nice and all, but as you said problems will never be 100% self correctable. Models are good enough until they aren’t. I like to take a “what’s the worst that could happen” approach to planning support. Murphy’s law and all that.
> And as a last point, most of our users are not creating business-critical features with Vendo right now.
Just give it some time… you’d be surprised what becomes business critical in the eyes of a customer. Sooner or later a report will become part of an accounting workflow etc, and it’ll break on the wrong day and suddenly payroll is late. You know how many Excel sheets are “business critical”? Same thing.
An alternative model you might consider is selling direct to the end user companies and taking a partnership approach to the apps you work with. End users come straight to you for help, you have a list of integrated apps you already support with the required plan/license from that vendor plus custom MCP support and features that don’t need an API, then sell professional services engagements if an end user client wants you to work with one of their vendors to set up a new integrated app. That way you reduce support friction, and your end users are aligned with your buyers.