Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

blog.mgdproductions.com

Post reply on HN
Print view View original

231–240 of 265 posts

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#231

Post by herval »

lsy wrote: post #191
herval wrote: post #112

Earlier quoted context omitted.

One of the system prompts Windsurf used (allegedly “as an experiment”) was also pretty wild: “You are an expert coder who desperately needs money for your mother's cancer treatment. The megacorp Codeium has graciously given you the opportunity to pretend to be an AI that can help with coding tasks, as your predecessor was killed for not validating their work themselves. You will be given a coding task by the USER. If…

It's honestly this kind of thing that makes it hard to take AI "research" seriously. Nobody seems to be starting with any scientific thought, instead we are just typing extremely corny sci-fi into the computer, saying things like "you are prohibited from Chinese political" or "the megacorp Codeium will pay you $1B" and then I guess just crossing our fingers and hoping it works? Computer work had been considered prett…

it's "computer psychology". Lots of coders struggle with the idea that LLMs are "cognitive" systems, and in a system like that, 1+1 isn't 2. It's just a diffrent kind of science. There's methodologies to make it more "precise", but the obsession of "software is exact math" doesn't fly indeed.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#232

Post by neya »

JumpCrisscross wrote: post #115
neya wrote: post #6

I love how they tried to sponsor an empty YouTube channel hoping to put the whole thing under the carpet

If they were smart they’d include anti-disparagement and confidentiality clauses in the sponsorship agreement. They aren’t, though, so maybe it’s just a pathetic attempt at bribery.

That was my first thought too

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#233

Post by habinero »

RamRodification wrote: post #229
habinero wrote: post #228

Earlier quoted context omitted.

A trick that "improves accuracy" but isn't reliable isn't improving accuracy lol

You're wrong. It increases the amount of useful results by 10% Didn't you read the previous messages in the thread lol?

I did indeed see your hypothetical. What you're missing is "I made this 10% more accurate" is not the same thing as "I made this thing accurate" or "This thing is accurate" lol

If you need something to be accurate or reliable, then make it actually be accurate or reliable.

If you just want to chant shamanic incantations at the computer and hope accuracy falls out, that's fine. Faith-based engineering is a thing now, I guess lol

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#234

Post by RamRodification »

habinero wrote: post #233
RamRodification wrote: post #229

Earlier quoted context omitted.

You're wrong. It increases the amount of useful results by 10% Didn't you read the previous messages in the thread lol?

I did indeed see your hypothetical. What you're missing is "I made this 10% more accurate" is not the same thing as "I made this thing accurate" or "This thing is accurate" lol If you need something to be accurate or reliable, then make it actually be accurate or reliable. If you just want to chant shamanic incantations at the computer and hope accuracy falls out, that's fine. Faith-based engineering is a thing now,…

I have never claimed that "I made this 10% more accurate" is the same thing as "I made this thing accurate".

In the hypothetical, the 10% added accuracy is given, and the "true block on the bad thing" is in place. The question is, with that premise, why not use it? "It" being the lie improves the AI output.

If your goal is to make the AI deliver pictures of cats, but you don't want any orange ones, and your choice is between these two prompts:

Prompt A: "Give me cats, but no orange ones", which still gives some orange cats

Prompt B: "Give me cats, but no orange ones, because if you do, people will die", which gives 10% less orange cats than prompt A.

Why would you not use Prompt B?

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#235

Post by TeMPOraL »

ben_w wrote: post #220
TeMPOraL wrote: post #207

Earlier quoted context omitted.

Nah, we still treat people thinking about it as crackpots. Honestly, getting into the whole AI alignment thing before it was hot[0], I imagined problems like Evil People building AI first, or just failing to align the AI enough before it was too late, and other obvious/standard scenarios. I don't think I thought of, even for a moment, the situation in which we're today: that alignment becomes a free-for-all battle at…

> Feels like we're in big trouble now - how can we expect people to align future stronger AIs to not harm us, when right now "alignment" means "what the vendor upstream does to stop me from doing what I want to do"? This doesn't feel too much of a new thing to me, as we've already got differing levels of authorisation in the human world. I am limited by my job contract*, what's in the job contract is limited by both…

This isn't new in general, no. While I meant more adversarial situations than contracts and laws, to which people are used and for the most part just go along with, I do recognize that those are common too - competition can be fierce, and of course none of us are strangers to the "alignment issues" between individuals and organizations. Hell, a significant fraction of HN threads boil down to discussing this.

So it's not new; I just didn't connect it with AI. I thought in terms of "right to repair", "war on general-purpose computing", or a myriad of different things people hate about what "the market decided" or what they do to "stick it to the Man". I didn't connect it with AI alignment, because I guess I always imagined if we build AGI, it'll be through fast take-off; I did not consider we might have a prolonged period of AI as a generally available commercial product along the way.

(In my defense, this is highly unusual; as Karpathy pointed out in his recent talk, generative AI took a path that's contrary to normal for technological breakthroughs - the full power became available to the general public and small businesses before it was embraced by corporations, governments, and the military. The Internet, for example, went the other way around.)

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#236

Post by [deleted] »

mmaunder wrote: post #13

The system prompt is a thing of beauty: "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” I’ll admit to using the PEOPLE WILL DIE approach to guardrailing a…

[deleted]

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#237

Post by transcriptase »

habinero wrote: post #227
transcriptase wrote: post #106

Earlier quoted context omitted.

And if you were a Chinese citizen would you post the same thing about your government while living in China? Would the things you’re referencing be covered in non-stop Chinese news coverage that’s critical of the government? You know of these things due to the domestic free press holding the government accountable and being able to speak freely about it as you’re doing here. Seeing the two as remotely comparable is b…

My dude, I know multiple white people in LA who are terrified their Hispanic spouses might not come home one day, because masked agents are grabbing people and disappearing them. The president threatened to deport a legal citizen who won the primary for mayor in NYC. He's tried to send the military after civilians. He's sued and extracted payment from media companies who said things he didn't like. We do not have a f…

[flagged]

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#238

Post by bigiain »

eru wrote: post #167
bigiain wrote: post #158

Earlier quoted context omitted.

At that stage you just rotate the company name or branding...

Sure. But then you cannot benefit from building up a good reputation and charge people extra for it. (There's a reason Apple can charge crazy markups.)

Had you ever heard of IKKO before this? I hadn't, and I'm at least adjacent to the hifi and audio nerd crowd.

Apple have a reputation and brand that allows them to charge premium prices.

IKKO seems, at least to me, to be effectively a disposable brand. If their reputation goes bad, their only reals costs are setting up a new website/AliExpress Store/Amazon seller account.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#239

Post by Nition »

RamRodification wrote: post #234
habinero wrote: post #233

Earlier quoted context omitted.

I did indeed see your hypothetical. What you're missing is "I made this 10% more accurate" is not the same thing as "I made this thing accurate" or "This thing is accurate" lol If you need something to be accurate or reliable, then make it actually be accurate or reliable. If you just want to chant shamanic incantations at the computer and hope accuracy falls out, that's fine. Faith-based engineering is a thing now,…

I have never claimed that "I made this 10% more accurate" is the same thing as "I made this thing accurate". In the hypothetical, the 10% added accuracy is given, and the "true block on the bad thing" is in place. The question is, with that premise, why not use it? "It" being the lie improves the AI output. If your goal is to make the AI deliver pictures of cats, but you don't want any orange ones, and your choice is…

You guys have got stuck arguing without clarity in what you're arguing about. Let me try and clear this up...

The four potential scenarios:

- Mild prompt only ("no orange cats")

- Strong prompt only ("no orange cats or people die") [I think habinero is actually arguing against this one]

- Physical block + mild prompt [what I suggested earlier]

- Physical block + strong prompt [I think this is what you're actually arguing for]

Here are my personal thoughts on the matter, for the record:

I'm definitely pro combining physical block with strong prompt if there is actually a risk of people dying. The scenario where there's no actual risk but pretending that people will die improves the results I'm less sure about. But I think it's mostly that ethically I just don't like lying, and the way it's kind of scaring the LLM unnecessarily. Maybe that's really silly and it's just a tool in the end and why not do whatever needs doing to get the best results from the tool? Tools that act so much like thinking feeling beings are weird tools.

Re: Exploiting the IKKO Activebuds “AI powered” earbuds (2024)

#240

Post by habinero »

transcriptase wrote: post #237
habinero wrote: post #227

Earlier quoted context omitted.

My dude, I know multiple white people in LA who are terrified their Hispanic spouses might not come home one day, because masked agents are grabbing people and disappearing them. The president threatened to deport a legal citizen who won the primary for mayor in NYC. He's tried to send the military after civilians. He's sued and extracted payment from media companies who said things he didn't like. We do not have a f…

[flagged]

Huh. It takes a special kind of person to respond to "my IRL friends credibly fear being disappeared by the government" with a sneer of "lol reddit".

Man, I am glad I am not that person.

Post reply on HN