Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I wouldn't say Pangram is broken, but I would say that it's brittle

freddiedeboer.substack.com

Post reply on HN
Print view View original

1–10 of 45 posts

Re: I wouldn't say Pangram is broken, but I would say that it's brittle

#3

Post by embedding-shape »

> If I want to induce a false positive in TSA's airport scanner, I can put a gun-shaped object in my bag. If I want to induce a false positive in Waymo's stop sign detection system, I can paint my own sign and put it up on a pole.

Holy strawman-batman, not only does the founder of Pangram not have a proper response to the actual criticisms, he feels the need to completely make up very different situations to try to illustrate some completely different point... I guess good job of the founder to engage at all, as deBoer does bring up a lot of valid points and criticisms of why people really shouldn't rely on "tools" like Pangram, too bad the founder failed completely at addressing the more serious points, and instead just chose to say "Well, there will be false-positives, what can you do?".

Re: I wouldn't say Pangram is broken, but I would say that it's brittle

#5

Post by cgio »

embedding-shape wrote: post #3

> If I want to induce a false positive in TSA's airport scanner, I can put a gun-shaped object in my bag. If I want to induce a false positive in Waymo's stop sign detection system, I can paint my own sign and put it up on a pole. Holy strawman-batman, not only does the founder of Pangram not have a proper response to the actual criticisms, he feels the need to completely make up very different situations to try to i…

Indeed, that’s not about false positives predominantly. At least for me the key concern is if it’s giving a percentage when it looks like it calculates a Boolean and therefore implies more nuanced analysis than it does. Which explains the nesting behaviour too.

Re: I wouldn't say Pangram is broken, but I would say that it's brittle

#6

Post by timpera »

I recently tried Pangram, created an account, wrote a few lines about how my day went, and it was flagged as likely AI-assisted. It clearly doesn't work.

It's especially bad that they keep insisting that it works very well, because thousands of people will probably end up falsely accused of AI usage as a result.

Re: I wouldn't say Pangram is broken, but I would say that it's brittle

#7

Post by skippyfish »

timpera wrote: post #6

I recently tried Pangram, created an account, wrote a few lines about how my day went, and it was flagged as likely AI-assisted. It clearly doesn't work. It's especially bad that they keep insisting that it works very well, because thousands of people will probably end up falsely accused of AI usage as a result.

Can you share your experiment, so that we can draw our own conclusions? I've been paying for Pangram for a long time and have not seen it misbehave even once. I've seen many people attack it, but when I looked more closely, their claims often evidently boiled down to "I want plausible deniability for my own use of LLMs" - i.e., they were clearly posting AI-generated content on social media and just didn't like the possibility of being called out.

I'm not saying that's you, but since a proof is easy to produce, it would be nice if you could share.

Re: I wouldn't say Pangram is broken, but I would say that it's brittle

#8

Post by ameliaquining »

embedding-shape wrote: post #3

> If I want to induce a false positive in TSA's airport scanner, I can put a gun-shaped object in my bag. If I want to induce a false positive in Waymo's stop sign detection system, I can paint my own sign and put it up on a pole. Holy strawman-batman, not only does the founder of Pangram not have a proper response to the actual criticisms, he feels the need to completely make up very different situations to try to i…

I'm confused, what are the "actual criticisms" you think weren't responded to?
Post reply on HN