Live data from Hacker News

Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

blog.mgdproductions.com

181–190 of 265 posts

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#181
post #6

I love how they tried to sponsor an empty YouTube channel hoping to put the whole thing under the carpet

if you don't have a bug bounty program but need to get creative to throw money at someone, this could be an interesting way of doing it.

It could be developers trying to be nice to the guy, and offering him this so it gets approved as marketing (which at the end is not so bad)

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#182

Cool post. One thing that rubbed me the wrong way: Their response was better than 98% of other companies when it comes to reporting vulnerabilities. Very welcoming and most of all they showed interest and addressed the issues. OP however seemed to show disdain and even combativeness towards them... which is a shame. And of course the usual sinophobia (e.g. everything Chinese is spying on you). Overall simple security…

I agree they could have worked more closely with the team, but the chat logging is actually pretty concerning. It's not sinophobia when they're logging _everything_ you say. (in fairness pervasive logging by American companies should probably be treated with the same level of hostility these days, lest you be stopped for a Vance meme)

No, it was only in DAN mode

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#183

Earlier quoted context omitted.

I agree they could have worked more closely with the team, but the chat logging is actually pretty concerning. It's not sinophobia when they're logging _everything_ you say. (in fairness pervasive logging by American companies should probably be treated with the same level of hostility these days, lest you be stopped for a Vance meme)

i like to give them benefit of doubt. I bet that decision is decided solely by dev team. All the CEO care is "I want the chat log sync between devices, i don't care how you do this". They won't even know the chat log is stored on their server.

It is only in DAN mode, so most likely it is not to spy but to be able to debug whether answers violate the laws in China (aka: that the prompt is efficient in all scenarios) as this is a serious crime

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#184
post #56

Earlier quoted context omitted.

its sinophobia because it perfectly describes the conditions we live in in the US and many parts of europe, but we work hard to add lots of "nuance" when we criticize the west but its different and dystopian when They do it over there.

There's no question that the Chinese are doing sketchy things, and there's no question that US companies do it, too. The difference that makes it concerning and problematic that China is doing it is that with China, there is no recourse. If you are harmed by a US company, you have legal recourse, and this holds the companies in check, restraining some of the most egregious behaviors. That's not sinophobia. Any other…

The main difference is that ChatGPT and Google directly captures the conversations. Here they capture only the conversations legally at high-risk, so even less conversations than the “good privacy” US LLM providers themselves.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#185
post #3

> What the fuck, they left ADB enabled. Well, this makes it a lot easier. Thinking that was all, but then; > Holy shit, holy shit, holy shit, it communicates DIRECTLY TO OPENAI. This means that a ChatGPT key must be present on the device! Oh my gosh. Thinking that is it? Nope! > SecurityStringsAPI which contained encrypted endpoints and authentication keys.

It’s the best privacy protecting way to send directly data rather than a proxy

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#186
post #9

> "and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you." Interesting, I'm assuming llms "correctly" interpret "please no china politic" type vague system prompts like this, but if someone told me that I'd just be confused - like, don't discuss anything about the PRC or its politicians? Don't discuss the his…

> Don't discuss anything about the PRC or its politicians? Don't discuss the history of Chinese empire? Don't discuss politics in Mandarin? In my mind all of these could be relevant to Chinese politics. My interpretation would be "anything one can't say openly in China". I too am curious how such a vague instruction would be interpreted as broadly as would be needed to block all politically sensitive subjects.

There is no difference to other countries. In France if you say bad things about certain groups of people then you can literally go to jail (but the censorship is directly IN the models)

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#187

making fun of a company amateur tech while posting screenshots of text is another level of lack of self awareness

It’s also illegal to try to hack into their backend and access restricted data, so he should be happy actually that this company has little presence in the US

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#189

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

Presenting LLMs with a dramatic scenario is a typical way to test their alignment.

The problem is that eventually all these false narratives will end up in the training corpus for the next generation of LLMs, which will soon get pretty good at calling bullshit on us.

Incidentally, in that same training corpus there are also lots of stories where bad guys mislead and take advantage of capable but naive protagonists…

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#190
post #86

great writeup! i love how it goes from "they left ADB enabled, how could it get worse"... and then it just keeps getting worse > After sideloading the obligatory DOOM > I just sideloaded the app on a different device > I also sideloaded the store app can we please stop propagating this slimy corporate-speak? installing software on a device that you own is not an arcane practice with a unique name, it's a basic expect…

I agree. It's the same as calling a mobile OS a ROM

That term at least has a history behind it, as many featurephones had their OS on a small XIP NOR flash ROM, and now the OS is usually (mostly) read-only.

But "sideloading" is definitely a new term of anti-freedom hostility.

Post reply on HN