Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

blog.mgdproductions.com

Post reply on HN
Print view View original

211–220 of 265 posts

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#211

Post by TeMPOraL »

frereubu wrote: post #141
pixelready wrote: post #81

Earlier quoted context omitted.

The irony of this is because it’s still fundamentally just a statistical text generator with a large body of fiction in its training data, I’m sure a lot of prompts that sound like terrifying skynet responses are actually it regurgitating mashups of Sci-fi dystopian novels.

Maybe this is something you heard too, but there was a This American Life episode where some people who'd had early access to what became one of the big AI chatbots (I think it was ChatGPT), but before they'd made it "nice", where they were asking it metaphysical questions about itself, and it was coming back with some pretty spooky answers and I was kind of intrigued about it. But then someone in the show suggested…

There are good analogies to be had in mythologies and folklore, too! Before there was science fiction - hell, even before there was science - people still occasionally thought of these things[0]. There are stories of deities and demons and fantastical creatures that explore the same problems AI presents - entities with minds and drives different to ours, and often possessing some power over us.

The arguably most basic and well-known example are entities granting wishes. The genie in Alladin's lamp, or the Goldfish[1]; the Devil in Faust, or in Pan Twardowski[2]. Variants of those stories go in detail over things we now call "alignment problem", "mind projection fallacy", "orthogonality thesis", "principal-agent problems", "DWIM", and others. And that's just scratching the surface; there's tons more in all folklore.

Point being - there's actually decent amount of thought people put into these topics over the past couple millennia - it's just all labeled religion, or folklore, or fairytale. Eventually though, I think more people will make a connection. And then the AI will too.

--

As for current generative models getting spooky, there's something else going on as well; https://www.astralcodexten.com/p/the-claude-bliss-attractor has a hypothesis I agree with.

--

[0] - For what reason? I don't know. Maybe it was partially to operationalize their religious or spiritual beliefs? Or maybe the storytellers just got there by extrapolating an idea in a logical fashion, following it to its conclusion. (which is also what good sci-fi authors do).

I also think the moment people started inventing spirits or demons that are more powerful than humans in some, but not all ways, some people started figuring out how use those creatures for their own advantage - whether by taming or tricking them. I guess it's human nature - when we stop fearing something, we think of how to exploit it.

[1] - https://en.wikipedia.org/wiki/The_Tale_of_the_Fisherman_and_... - this is more of a central/eastern Europe thing.

[2] - https://en.wikipedia.org/wiki/Pan_Twardowski - AKA the "Polish Faust".

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#212

Post by efilife »

userbinator wrote: post #190
efilife wrote: post #134

Earlier quoted context omitted.

I agree. It's the same as calling a mobile OS a ROM

That term at least has a history behind it, as many featurephones had their OS on a small XIP NOR flash ROM, and now the OS is usually (mostly) read-only. But "sideloading" is definitely a new term of anti-freedom hostility.

I expected someone to say this. In my opinion if there's some history behind something stupid it doesn't make it not stupid

Btw interesting stats here https://trends.google.com/trends/explore?date=all&q=%2Fm%2F0...

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#213

Post by komali2 »

throw10920 wrote: post #210
komali2 wrote: post #196

Earlier quoted context omitted.

You don't feel there's a difference between a State banning criticism of the State, and a State passing anti-hate speech laws to protect people from, e.g., nazis?

No, there isn't a difference. "Hate speech" has no meaning, and laws purporting to be combatting it are actively used to prevent criticism of the State (e.g. in Germany).

This is strange to me. I have no difficulty seeing the difference between hate speech and criticism of the state. Of course if someone tries to muddy the waters, they should be criticized... but that's what you're trying to do here, so you're no better than a State that does the same. Hate speech very clearly has meaning, the legal definition may change a bit of course, but in Germany the meaning is quite clear, banning expressions that incite hatred or violence against people based on race, ethnicity, religion, nationality, gender, sexual orientation, or disability. What's unclear about that?

I'm not sure what specific incident you're referring to, however I do know that if Germany was more willing to leverage the hate speech laws more strictly, the AFD would have been banned long ago. Now they're finally willing to leverage it to ban the new nazi party, which is a relief.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#214

Post by throwawayoldie »

lxe wrote: post #59

That's some very amateur programming and prompting that you've exposed.

When the ZIRP era ended, I thought it would turn out to be a good thing for the industry, since it would wash out a lot of lightweights and incompetents.

Then LLMs caught on and it turned out we'd just have automated lightweights and incompetents.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#215

Post by yellow_postit »

herval wrote: post #112
mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

One of the system prompts Windsurf used (allegedly “as an experiment”) was also pretty wild: “You are an expert coder who desperately needs money for your mother's cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If…

Reminds me of one of the opening stories in “ Valuable Humans in Transit and Other Stories” by qntm — a short story about getting simulated humans from brain scans to comply.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#216

Post by throw10920 »

komali2 wrote: post #213
throw10920 wrote: post #210

Earlier quoted context omitted.

No, there isn't a difference. "Hate speech" has no meaning, and laws purporting to be combatting it are actively used to prevent criticism of the State (e.g. in Germany).

This is strange to me. I have no difficulty seeing the difference between hate speech and criticism of the state. Of course if someone tries to muddy the waters, they should be criticized... but that's what you're trying to do here, so you're no better than a State that does the same. Hate speech very clearly has meaning, the legal definition may change a bit of course, but in Germany the meaning is quite clear, bann…

> I have no difficulty seeing the difference between hate speech and criticism of the state.

You have no difficulty manufacturing what you believe to be a difference (that clearly does not survive contact with reality), because you're ignorant of the world around you.

> Of course if someone tries to muddy the waters, they should be criticized

No, if someone tries to falsely claim that there's a clear and objective difference, as you are, they should be criticized.

> Hate speech very clearly has meaning

No, it very clearly does not, and the fact that you're expressing that opinion indicates that you're extremely uninformed about history. "Hate speech" wasn't even a concept that existed until the 20th century, originally only referred to race when it was defined by the ICERD, constantly changed and increased in scope, and still even today not only has no commonly agreed-upon definition, but is used to suppress relevant-to-society free speech that the State does not approve of.

If you go and ask 10 random people in your country what the definition of "hate speech" is, they will not be able to agree on a definition - anyone who has gone out and actually interacted with different groups in their country (as opposed to being isolated to a single community) knows this to be true. That by itself is factual proof that there is no consensus definition of the term.

Not that there needs to be any further elaboration than that, but...

> I'm not sure what specific incident you're referring to

Marie-Thérèse Kaiser, a German politician, posted a social media post with the text "Afghanistan refugees; Hamburg SPD mayor for 'unbureaucratic' admission; Welcome culture for gang rapes?" and was charged under German hate speech laws. You're extremely authoritarian and progressive, so you probably feel that a penalty should have been given out, but regardless of your feelings, the fact is that that was not clearly incitement to hatred or violence, and that the poster was charged for "hate speech" for making political statements about immigration.

> banning expressions that incite hatred or violence against people based on [...]. What's unclear about that?

It's very clear to anyone who has contact with reality that not only does "hatred" also have no consensus definition, but neither does "inciting", and so both of those terms can be and are interpreted in an extremely wide spread that is abused by the State.

Not only is the lack of consensus of definition of the concept of "hate speech" factual evidence that your claims about it being clear are false, but even your citation of the German legal definition contains terms that have neither consensus population definition nor objective test (legal or otherwise).

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#217

Post by peab »

mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

A serious person won't use a system prompt as guardrails against a system that would have direct real world consequences of people dying like that. They'd have failsafes baked in

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#218

Post by [deleted] »

komali2 wrote: post #196
rvnx wrote: post #186

Earlier quoted context omitted.

There is no difference to other countries. In France if you say bad things about certain groups of people then you can literally go to jail (but the censorship is directly IN the models)

You don't feel there's a difference between a State banning criticism of the State, and a State passing anti-hate speech laws to protect people from, e.g., nazis?

[deleted]

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#219

Post by pxc »

pxc wrote: post #192
elashri wrote: post #22

Earlier quoted context omitted.

From my experience (which might be incorrect) LLMs find hard time recognize how many words they will spit as response for a particular prompt. So I don't think this work in practice.

Indeed, it doesn't work. LLMs can't count. They have no need of how many words they've used. If you ask an LLM to track how many words or tokens it has used in a conversation, it will roleplay such counting with totally bullshit numbers.

oops, "no need" should be "no notion"

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#220

Post by ben_w »

TeMPOraL wrote: post #207
xandrius wrote: post #77

Earlier quoted context omitted.

Not sad, before it was SciFi and now we are actually thinking about it.

Nah, we still treat people thinking about it as crackpots. Honestly, getting into the whole AI alignment thing before it was hot[0], I imagined problems like Evil People building AI first, or just failing to align the AI enough before it was too late, and other obvious/standard scenarios. I don't think I thought of, even for a moment, the situation in which we're today: that alignment becomes a free-for-all battle at…

> Feels like we're in big trouble now - how can we expect people to align future stronger AIs to not harm us, when right now "alignment" means "what the vendor upstream does to stop me from doing what I want to do"?

This doesn't feel too much of a new thing to me, as we've already got differing levels of authorisation in the human world.

I am limited by my job contract*, what's in the job contract is limited by both corporate requirements and the law, corporate requirements are also limited by the law, the law is limited by constitutional requirements and/or judicial review and/or treaties, treaties are limited by previous and foreign governments.

* or would be if I was working; fortunately for me in the current economy, enough passive income that my savings are still going up without a job, plus a working partner who can cover their own share.

Post reply on HN