Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting
Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
21–30 of 67 posts
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#22We’ll likely see a lot of these AI pet projects get axed in the coming year or two… especially things rushed out in the early phases of the AI bubble when folks were desperate to appear to be using AI.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#23Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
Why do you think OpenAI let a red team loose on GPT-5 for six months before releasing it to the public?
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#24We’ll likely see a lot of these AI pet projects get axed in the coming year or two… especially things rushed out in the early phases of the AI bubble when folks were desperate to appear to be using AI.
yeah i hope the problems stay to somewhat humorous themes like convincing a car sales bot to sell you a car for $1 and not more serious issues like convincing a bot to metaphorically launch the ICBMs.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#25Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
> Why did NYC release it in the first place? Did they not QA it? Considering Louis Rossmann's videos on his adventures with NYC bureaucracy (e.g. [0]), the QAers might not have known the laws any better than the chat bot. [0] https://www.youtube.com/watch?v=yi8_9WGk3Ok
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#26Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting
The thing is (and maybe this is what parent meant by non-determinism, in which case I agree it's a problem), in this brave new technological use-case, the space of possible interactions dwarfs anything machines have dealt with before. And it seems inevitable that the space of possible misunderstandings which can arise during these interactions will balloon similarly. Simply because of the radically different nature of our AI interlocutor, compared to what (actually, who) we're used to interacting with in this world of representation and human life situations.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#27Being in and around the NYC area, while also knowing plenty of small businesses, I'm glad Mamdani killed this bot. Telling bosses to steal tips from their employees is run-of-the-mill corruption and common over here. The vibe for businesses is that everyone has to be exploiting someone else or have a schtick. If you were to talk about morals, you would be ridiculed. Most lawyers wouldn't even prosecute small business…
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#28Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
> Why did NYC release it in the first place? Perhaps a big fat check was involved.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#29Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
I'm sure they QA'd it, but QA was probably "does this give me good results" (almost certainly 'yes' with an LLM), not "does this consistently not give me bad results".