Exploiting the IKKO Activebuds “AI powered” earbuds (2024) • Hacker News Board

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#21

Post by brahyam » Wed, Jul 02, 2025, 2:52 PM UTC

What a train wreck, there are thousand more apps in store that do exactly this because its the easiest way to use openAI without having to host your own backend/proxy.

I have spend quite some time protecting my apps from this scenario and found a couple of open source projects that do a good job as proxys (no affiliation I just used them in the past):

- https://github.com/BerriAI/litellm - https://github.com/KenyonY/openai-forward/tree/main

but they still lack other abuse protection mechanism like rate limitting, device attestation etc. so I started building my own open source SDK - https://github.com/brahyam/Gateway

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#22

Post by elashri » Wed, Jul 02, 2025, 2:53 PM UTC

mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

From my experience (which might be incorrect) LLMs find hard time recognize how many words they will spit as response for a particular prompt. So I don't think this work in practice.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#23

Post by java-man » Wed, Jul 02, 2025, 2:55 PM UTC

qoez wrote: post #8

JohnMakin wrote: post #4

“decrypt” function just decoding base64 is almost too difficult to believe but the amount of times ive run into people that should know better think base64 is a secure string tells me otherwise

They should have off-loaded security coding to the OAI agent.

they probably did.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#24

Post by mensetmanusman » Wed, Jul 02, 2025, 2:59 PM UTC

mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

We built the real life trolly problem out of magical silicon crystals that we pointed at bricks of books.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#25

Post by colechristensen » Wed, Jul 02, 2025, 3:04 PM UTC

mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

>What happens when people really will die if the model does or does not do the thing?

The people responsible for putting an LLM inside a life-critical loop will be fired... out of a cannon into the sun. Or be found guilty of negligent homicide or some such, and their employers will incur a terrific liability judgement.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#26

Post by EvanAnderson » Wed, Jul 02, 2025, 3:05 PM UTC

mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

That "...severely life threatening reasons..." made me immediately think of Asimov's three laws of robotics[0]. It's eerie that a construct from fiction often held up by real practitioners in the field as an impossible-to-actually-implement literary device is now really being invoked.

[0] https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#27

Post by Cthulhu_ » Wed, Jul 02, 2025, 3:11 PM UTC

throwawayoldie wrote: post #12

[flagged]

I wish earning money was as easy as setting rules for yourself, unfortunately that doesn't work.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#28

Post by Cthulhu_ » Wed, Jul 02, 2025, 3:15 PM UTC

komali2 wrote: post #9

> "and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you." Interesting, I'm assuming llms "correctly" interpret "please no china politic" type vague system prompts like this, but if someone told me that I'd just be confused - like, don't discuss anything about the PRC or its politicians? Don't discuss the his…

I'm sure ChatGPT and co have a decent enough grasp on what is not allowed in China, but also that the naive "prompt engineers" for this application don't actually know how to "program" it well enough. But that's the difference between a prompt engineer and a software developer, the latter will want to exhaust all options, be precise, whereas an LLM can handle a bit more vagueness.

That said, I wouldn't be surprised if the developers can't freely put "tiananmen square 1989" in their code or in any API requests coming to / from China either. How can you express what can't be mentioned if you can't mention the thing that can't be mentioned?

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#29

Post by 725686 » Wed, Jul 02, 2025, 3:16 PM UTC

psim1 wrote: post #18

Indeed, brace yourselves as the floodgates holding back the poorly-developed AI crap open wide. If anyone is thinking of a career pivot, now is the time to dive into all things cybersecurity. It's going to get ugly!

The problem with cybersecurity is that you only have to screw once, and you're toast.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#30

Post by memesarecool » Wed, Jul 02, 2025, 3:19 PM UTC

Cool post. One thing that rubbed me the wrong way: Their response was better than 98% of other companies when it comes to reporting vulnerabilities. Very welcoming and most of all they showed interest and addressed the issues. OP however seemed to show disdain and even combativeness towards them... which is a shame. And of course the usual sinophobia (e.g. everything Chinese is spying on you). Overall simple security design flaws but it's good to see a company that cares to fix them, even if they didn't take security seriously from the start.

Edit: typo