He is turning out to be a benevolent, law-abiding mayor that just happens to be communist.
What's that supposed to mean?
Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
31–40 of 67 posts
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#32Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#33Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
Remember that many people are heavily are happy-path biased. They see a good result once and say "that's it, ship it!" I'm sure they QA'd it, but QA was probably "does this give me good results" (almost certainly 'yes' with an LLM), not "does this consistently not give me bad results".
> almost certainly 'yes' with an LLM
LLMs can handle search because search is intentionally garbage now and because they can absorb that into their training set.
Asking highly specific questions about NYC governance, which can change daily, is almost certainly 'not' going to give you good results with an LLM. The technology is not well suited to this particular problem.
Meanwhile if an LLM actually did give you good results it's an indication that the city is so bad at publishing information that citizens cannot rightfully discover it on their own. This is a fundamental problem and should be solved instead of layering a $600k barely working "chat bot" on top the mess.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#34Earlier quoted context omitted.
>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting
The same way you test any system - you find a sampling of test subjects, have them interact with the system and then evaluate those interactions. No system is guaranteed to never fail, it's all about degree of effectiveness and resilience. The thing is (and maybe this is what parent meant by non-determinism, in which case I agree it's a problem), in this brave new technological use-case, the space of possible interac…
[deleted]
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#35Earlier quoted context omitted.
>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting
The same way you test any system - you find a sampling of test subjects, have them interact with the system and then evaluate those interactions. No system is guaranteed to never fail, it's all about degree of effectiveness and resilience. The thing is (and maybe this is what parent meant by non-determinism, in which case I agree it's a problem), in this brave new technological use-case, the space of possible interac…
> radically different nature of our AI interlocutor
It's the training data that matters. Your "AI interlocutor" is nothing more than a lossy compression algorithm.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#36> A spokesperson for the mayor, Dora Pekec, confirmed in a text message that the new administration plans to take down the chatbot. She said a member of the Mamdani transition team had seen reporting on the bot from The Markup and THE CITY and presented it to the mayor as a possible place to save funds. Journalism works.
Journalism teed up an easy way for an incoming politician to dunk on his predecessor, if you'll forgive the mixed metaphor. Not that I'm opposed to any part of it, just that this was an easy scenario for "journalism" to "work" in.
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#37Earlier quoted context omitted.
yeah i hope the problems stay to somewhat humorous themes like convincing a car sales bot to sell you a car for $1 and not more serious issues like convincing a bot to metaphorically launch the ICBMs.
"The WOPR did a better job avoiding thermonuclear war than most humans would" is my hot take.
Thinks through possibilities -> realizes what it is proposing is a bad idea
Hell put WOPR in charge of everything
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#38Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?
> Why did NYC release it in the first place? Perhaps a big fat check was involved.
Yeah… no offense, but only a person who didn't know anything about Mayor Eric Adams would ask a question like that.
Just days out of office, he made a few million off a crypto scam. Buffoonishly corrupt. https://finance.yahoo.com/news/eric-adams-promoted-memecoin-...
Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#39Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law
#40Earlier quoted context omitted.
>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting
The same way you test any system - you find a sampling of test subjects, have them interact with the system and then evaluate those interactions. No system is guaranteed to never fail, it's all about degree of effectiveness and resilience. The thing is (and maybe this is what parent meant by non-determinism, in which case I agree it's a problem), in this brave new technological use-case, the space of possible interac…
Does knowing the system architecture not help you with defining things like happy path vs edge case testing? I guess it's much less applicable for overall system testing, but in "normal" systems you test components separately before you test the whole thing, which is not the case with LLMs.
By "non-deterministic" I meant that it can give you different output for the same input. Ask the same question, get a different answer every time, some of which can be accurate, some... not so much. Especially if you ask the same question in the same dialog (so question is the same but the context is not so the answer will be different).
EDIT: More interestingly, I find an issue, what do I even DO? If it's not related to integrations or your underlying data, the black box just gave nonsensical output. What would I do to resolve it?