Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

themarkup.org

Post reply on HN
Print view View original

51–60 of 67 posts

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#51

Post by themafia »

sebastiennight wrote: post #41
themafia wrote: post #35

Earlier quoted context omitted.

> radically different nature of our AI interlocutor It's the training data that matters. Your "AI interlocutor" is nothing more than a lossy compression algorithm.

Most AI Chatbots do not rely on their training data, but on the data that is passed to them through RAG. In that sense they are not compressing the data, just searching and rewording it for you.

> and rewording it

Using the probabilities encoded in the training data.

> In that sense they are not compressing the data

You're right. In this case they're decompressing it.

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#52

Post by themafia »

pegasus wrote: post #48
themafia wrote: post #35

Earlier quoted context omitted.

> radically different nature of our AI interlocutor It's the training data that matters. Your "AI interlocutor" is nothing more than a lossy compression algorithm.

Yet it won't be easy not to anthropomorphize it, expecting it to just know what we mean, as any human would. And most of the time it will, but once in a while it will betray its unthinking nature, taking the user by surprise.

> taking the user by surprise.

And surprise is really what you want in computing. ;)

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#53

Post by Neywiny »

sdwr wrote: post #47
Neywiny wrote: post #46

I always ask this question about these bots: is the literature the training data or is the understanding of literature the training data? Meaning, sure you trained the bot on the current rules and regulations. But does that mean the model weights contain them? Such that really is a guess at legal accuracy? Or is it trained to be a lawyer and understand the docs which sit outside the model? Every time I've asked the a…

> pretty easy to do what the Google AI does I thought Gemini just started providing citations in the last few months. Are you saying they should have beaten Google to the punch on this? As part of the $500,000 budget?

Correct. Much in the same way that videos were online before YouTube, social networks existed before Facebook, and messaging existed before WhatsApp and co, they should have understood their problem set better instead of just following the leaders. Because Gemeni is not this chatbot on steroids, it's a different problem entirely that happens to now employ the same technique.

Also, search says they did links in 2024 for the Google AI. So there's that.

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#55

Post by georgemcbay »

1970-01-01 wrote: post #32
direwolf20 wrote: post #31

Earlier quoted context omitted.

What's that supposed to mean?

The previous mayors were none of these things

Mamdani is a socialist, not a communist.

And Fiorello La Guardia was (in terms of beliefs and enacted policy) even more socialist than Mamdani is even though he was technically a Republican when elected.

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#56

Post by kylehotchkiss »

drillsteps5 wrote: post #17
andsoitis wrote: post #6

Why did NYC release it in the first place? Did they not QA it? Or was it perhaps one of those cases where they found issues, but the only way to really know for sure that the deleterious impact is significant enough by pushing it to prod?

>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting

temperature 0 and 10,000,000 mischievous prompts

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#57

Post by worthless-trash »

Eric_WVGG wrote: post #38
erxam wrote: post #7

Earlier quoted context omitted.

> Why did NYC release it in the first place? Perhaps a big fat check was involved.

Yeah… no offense, but only a person who didn't know anything about Mayor Eric Adams would ask a question like that. Just days out of office, he made a few million off a crypto scam. Buffoonishly corrupt. https://finance.yahoo.com/news/eric-adams-promoted-memecoin-...

Not op. That wasn't a question.

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#58

Post by nate_meurer »

toomuchtodo wrote: post #44
andrewflnr wrote: post #36

Earlier quoted context omitted.

Journalism teed up an easy way for an incoming politician to dunk on his predecessor, if you'll forgive the mixed metaphor. Not that I'm opposed to any part of it, just that this was an easy scenario for "journalism" to "work" in.

If you'd like other examples, 404media and adjacent journalism grinding against Flock across the country, as well as perfectunion working against datacenter siting. I admit the egregious nature of the Adams NYC administration and his fraud makes this particular scenario straightforward. https://en.wikipedia.org/wiki/Investigations_into_the_Eric_A...

Yes the Flock coverage is a very good example.

Re: Mamdani to kill the NYC AI chatbot caught telling businesses to break the law

#60

Post by datsci_est_2015 »

pegasus wrote: post #26
drillsteps5 wrote: post #17

Earlier quoted context omitted.

>Why did NYC release it in the first place? Did they not QA it? How do you QA black box non-deterministic system? I'm not being facetious, seriously asking. EDIT: Formatting

The same way you test any system - you find a sampling of test subjects, have them interact with the system and then evaluate those interactions. No system is guaranteed to never fail, it's all about degree of effectiveness and resilience. The thing is (and maybe this is what parent meant by non-determinism, in which case I agree it's a problem), in this brave new technological use-case, the space of possible interac…

> The same way you test any system - you find a sampling of test subjects, have them interact with the system and then evaluate those interactions.

That’s not strictly how I test my systems. I can release with confidence because of a litany of SWE best practices learned and borrowed from decades of my own and other people’s experiences.

> No system is guaranteed to never fail, it's all about degree of effectiveness and resilience.

It seems like the product space for services built on generative AI is diminishing by the day with respect to “effectiveness and resilience”. I was just laughing with some friends about how terrible most of the results are when using Apple’s new Genmoji feature. Apple, the company with one of the largest market caps in the world.

I can definitely use LLMs and other generative AI directly, and understand the caveats, and even get great results from them. But so far every service I’ve interacted with that was a “white label” repackaging of generative AI has been absolute dogwater.

Post reply on HN