Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I almost got hacked by a 'job interview'

blog.daviddodda.com

Post reply on HN
Print view View original

521–530 of 534 posts

Re: I almost got hacked by a 'job interview'

#521

Post by strbean »

acdha wrote: post #382
strbean wrote: post #254

Earlier quoted context omitted.

Eh, it would be nice if there was a public title database in the US. Ideally government administered, but if we can't have that then maybe a distributed ledger would do the trick. It's hilarious that title searches and title insurance exist. And even more ridiculous that there is just no way, period, to actually verify that a would-be landlord is actually authorized to lease you a place to live.

> Ideally government administered, but if we can't have that then maybe a distributed ledger would do the trick. The problem is that it has to be government administered because otherwise you’re constantly stuck with the risk that what you see won’t survive a legal challenge. This is a constant problem for ledgers because the sales pitch is about being “trust less” or distributed in some sense that everyone can parti…

Yeah, a distributed ledger for managing property ownership would be pretty borked with no authority to revert thefts, enforce legal orders, etc.

It might be an effective way to get buy in from the government if they don't have to manage much infrastructure, if they still get the (literal?) keys to intervene in things. That would require them to have the basic competency to manage their own access, though.

Re: I almost got hacked by a 'job interview'

#522

Post by mquander »

DavidDodda wrote: post #68
nubg wrote: post #9

This article was written by an LLM. I get that the author might be self-conscious about his English writing skills, but I would still much rather read the original prompt that the author put into ChatGPT, instead of the slop that came out. The story - if true - is very interesting of course. Big bummer therefore that the author decided to sloppify it. David, could you share as a response to this comment the original…

thanks for the feedback. just fyi - this went though 11 different versions before reaching this point. so I am not able to share the full chat because i used Claude with google docs integration. but hears the google doc i started with https://docs.google.com/document/d/1of_uWXw-CppnFtWoehIrr1ir... this and the following prompt ``` 'help me turn this into a blog post. keep things interesting, also make sure you take a…

This is fascinating, thanks so much for posting it!

Re: I almost got hacked by a 'job interview'

#523

Post by zahlman »

teo_zero wrote: post #353
DavidDodda wrote: post #68

Earlier quoted context omitted.

thanks for the feedback. just fyi - this went though 11 different versions before reaching this point. so I am not able to share the full chat because i used Claude with google docs integration. but hears the google doc i started with https://docs.google.com/document/d/1of_uWXw-CppnFtWoehIrr1ir... this and the following prompt ``` 'help me turn this into a blog post. keep things interesting, also make sure you take a…

> You are an AI Bot that is very good at mimicking an author writing style. - Your goal is to write content with the tone that is described below Genuine question: does this formulation style work better than a plain, direct "Mimick my writing style. Use the tone that is described below"?

Traditionally (he said, referring to prior art in a field that has basically only existed for about 3 years), this sort of "flattery" was understood (he said, referring to random rumours he'd read on the Internet) to make a big difference, since otherwise the LLM might roleplay as something else. Presumably the RLHF training is stronger now.

Re: I almost got hacked by a 'job interview'

#524

Post by zahlman »

mvdtnz wrote: post #434
Wowfunhappy wrote: post #233

This article is so interesting, but I can’t shake the feeling it was written by AI. The writing style has that feel for me. Maybe that shouldn’t bother me? Like, maybe the author would never have had time to write this otherwise, and I would never have learned about his experience. But I can't help wishing he'd just written about it himself. Maybe that's unreasonable--I shouldn't expect people to do extra work for fr…

It has many of the hallmarks of AI prose. It's amazing to me that people can't spot this stuff just by feel alone, * Not X. Not Y. Just Z. * The X? A Y. ("The scary part? This attack vector is perfect for developers.", "The attack vector? A fake coding interview from") * The X was Y. Z. (one-word adjectives here). * Here's the kicker. * Bullet points with a bold phrase starting each line. The weird thing is that befo…

My assumption is that people absolutely did, and do, write like that all the time. Just not necessarily in places that you'd normally read. LLM drags up idioms from all over its training set and spews them back everywhere else, without contextual awareness. (That also means it averages across global cultures by default.)

But also, over the last three years people have been using AI to output their own slop, and that slop has made its way back into the training data for later iterations of the technology.

And then there's the recent revelation (https://www.anthropic.com/research/small-samples-poison , which I got from HN) that it might not actually take a whole lot of examples in the data for an LLM to latch onto some pattern hard.

Re: I almost got hacked by a 'job interview'

#525

Post by zahlman »

tabbott wrote: post #449
Wowfunhappy wrote: post #233

This article is so interesting, but I can’t shake the feeling it was written by AI. The writing style has that feel for me. Maybe that shouldn’t bother me? Like, maybe the author would never have had time to write this otherwise, and I would never have learned about his experience. But I can't help wishing he'd just written about it himself. Maybe that's unreasonable--I shouldn't expect people to do extra work for fr…

Yeah my reaction was: - The class of threat is interesting and worth taking seriously. I don't regret spending a few minutes thinking about it. - The idea of specifically targeting people looking for Crypto jobs from sketchy companies for your crypto theft malware seems clever. - The text is written by AI. The whole story is a bit weird, so it's plausible this is a made up story written by someone paid to market Curs…

> be tested against popular LLMs, perhaps work by tricking Cursor and similar tools into installing the malware, without the user running anything themselves

My sense is that the attack isn't nearly as sophisticated as it looks, and the attackers out there aren't really thinking about things on this level — yet.

> Hide the shellcode in an `npm` dependency

It would have to be hidden specifically in a post-install script or similar. Which presumably isn't any harder, but.

Re: I almost got hacked by a 'job interview'

#526

Post by zahlman »

mensetmanusman wrote: post #381
Yondle wrote: post #327

Earlier quoted context omitted.

Your comment was so validating, I was getting such weird vibes and felt it was so dumbly written given the contention was actually good advice. Consequently, the author tarnished his reputation for me personally from the very beginning.

It’s easy to ask an llm to change writing styles though… this is what the dead internet feels like.

Per https://news.ycombinator.com/item?id=45594554 , OP actually did attempt a fair bit of customization, but the result was still unmistakable.

Re: I almost got hacked by a 'job interview'

#527

Post by zahlman »

andy99 wrote: post #299
rdtsc wrote: post #273

Earlier quoted context omitted.

> This article is so incredibly interesting, but I can’t shake the feeling it was written by AI. The writing style has all the telltale signs. The sadder realization is that after enough AI slop around, real people will start talking like AI. This will just become the new standard communication style.

Even now, I think many people are not literate enough to see that it’s bad, and in fact think it improves their writing (beyond just adding volume). Maybe that’s a good thing? It’s given a whole group of people who otherwise couldn’t write a voice (that of a contract African data labeller). Personally I still think it’s slop, but maybe in fact it is a kind of communication revolution? Same way writing used to only be…

If they aren't literate enough to see that it's bad, then it probably actually is an improvement over their own writing.

Re: I almost got hacked by a 'job interview'

#528

Post by zahlman »

citizenpaul wrote: post #417
wholinator2 wrote: post #317

Earlier quoted context omitted.

Very interesting idea. You could even take it a step farther and include multiple layers of string mixing. Though i imagine after a certain point the obfuscation to suspicion ratio shifts firmly in the direction of suspicion. I wonder what the sweet spot is there

Based on the complete out of my behind number I'd say something like 99.9999% of successful hacks I read about use one level of abstraction or less. Heavy emphasis on the less. So I think one layer of abstraction will get you pretty far with most targets.

If anything, the pattern of the obfuscated code is a red flag for both human and LLM readers (although of course the LLM will read much faster). You don't have to figure out what it does to know it's suspicious (although LLMs are better at that than I would have expected, and humans have a variety of techniques available to them).

Re: I almost got hacked by a 'job interview'

#529

Post by johnisgood »

chipsrafferty wrote: post #510
wizzwizz4 wrote: post #308

Earlier quoted context omitted.

People are often unconfident about their own writing. But if you can feed it to a LLM and have the LLM output something that looks coherent, your writing is good enough to publish.

If you publish the input instead, if other people feel it's not polished or whatever, they can run it through an LLM. What OP did was destroy value instead of create it, you can always run it through another LLM with another prompt if you have the input, but you can't go backwards.

Just like we have a light/dark switch, we will have a Raw/AI'd switch.

Re: I almost got hacked by a 'job interview'

#530

Post by johnisgood »

nytesky wrote: post #397
Wowfunhappy wrote: post #233

This article is so interesting, but I can’t shake the feeling it was written by AI. The writing style has that feel for me. Maybe that shouldn’t bother me? Like, maybe the author would never have had time to write this otherwise, and I would never have learned about his experience. But I can't help wishing he'd just written about it himself. Maybe that's unreasonable--I shouldn't expect people to do extra work for fr…

My daughter feels all my writing naturally sounds like AI, even my college papers from 30 years ago. Maybe author has similar issue?

I have been told I am "AI" because I was simply a bit too serious, enthusiastic and nerdy about some topic. It happens. I put more effort into such writings. Check my comment history and you will find that many comments from me are low-effort: including this one. :)
Post reply on HN