Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Profiling Hacker News users based on their comments

simonwillison.net

Post reply on HN
Print view View original

81–90 of 96 posts

Re: Profiling Hacker News users based on their comments

#81

Post by ethbr1 »

api wrote: post #37
alexpotato wrote: post #14

Story time: My first full time job (early 2000s) was working for a firm that did online cybersecurity related investigations for Fortune 500 companies (generally via a 3rd party law firm they had retained). A big part of this was running investigations into people running "pump and dump" stock schemes on Yahoo message boards. We would generally start by scraping all of the posts for a user who had instigated one of t…

Think about browser fingerprinting. Every little bit of info is literally one more bit, so by the time you get to 32 bits you’ve narrowed it down to one in four billion. An oversimplification but that’s the idea. Being strongly private online requires spy tradecraft levels of precaution.

Or just making certain topics verboten. Different pieces of information can be order of magnitude more or less useful.

Re: Profiling Hacker News users based on their comments

#82

Post by ethbr1 »

stego-tech wrote: post #55
simonw wrote: post #45

Earlier quoted context omitted.

Right, as is so often the case with AI stuff the thing that's disconcerting is how cheap and low friction and friction adopt available this ability is now. Anyone with access to a decent LLM can now perform a version of this in just a few seconds.

It's a lot to take in, if I'm being honest. Growing up in the sort of cultures where gossip and tabloids were the norm, this tool is painful to me in a way I'm not sure many folks can understand. It's not even low friction anymore; it's no friction, in the sense that anyone with a chatbot and minimal rails can just ask it to do these sorts of profiles now, on anyone they choose. We desperately need to modernize laws…

I think it's not unreasonable, if one is in an oft discriminated protected class, to aim ones career / expense trajectory towards stability for the next couple decades. (Prioritizing remote, focusing towards subfields where there's more tolerance, working for companies in financially stable industries)

The law, currently predicated on the difficulty of discriminating en masse without leaving a paper trail, will take a while to catch up with de facto use.

Hell, there's still no legal prohibition in most of the US on things like Equifax's salary history reporting. https://en.wikipedia.org/wiki/The_Work_Number

Re: Profiling Hacker News users based on their comments

#83

Post by michaelteter »

tasuki wrote: post #77
michaelteter wrote: post #47

And note that HN does not allow you to delete your comments after a short time passes. If you contact them and ask for your data to be deleted, they will directly refuse.

Yes it would break things. Also it's impossible to delete things from the internet, so a bit cute to attempt to do it...

The comment text could be replaced with (deleted), and the poster name could be as well.

And while other sites mirroring HN might keep the original, that’s a separate issue.

Re: Profiling Hacker News users based on their comments

#84

Post by sdoering »

janalsncm wrote: post #13

Not doubting the method works in general, but Simon Willison is a public-enough figure so the baseline level of info is higher than just HN comments. If you turn off Claude’s web search: > Simon Willison is a British software developer, blogger, and open-source advocate, best known for…

I just had Claude Code built the comment-fetching part as a CLI tool. And then use this data on my last 200 comments to profile me. In an empty context. Without web search.

Holy moly - it was damn accurate. Those comments spanned the last few year, as I became less and less active in social media/online comments in general. But it picked up on me being a snob about unprocessed food, having hard red lines that I sometimes take a very harsh stand over, as well as so many other things.

I know I am quite open in what I write on the net and do not really fear future employers using this to qualify me - because if they do and deny me a job I know I would not have wanted to work there in the first place. And yes, this is a position of old white grumpy male privilege.

Still. It was impressive and "quite a bit dystopian". All in all, from posting the URL to Simon's blog post to being profiled took less than 6 minutes.

Re: Profiling Hacker News users based on their comments

#85

Post by ethbr1 »

nirui wrote: post #64
stego-tech wrote: post #40

This is...disquieting. It's one thing to know that it's possible, another thing to know nation states or large megacorps are doing it, but another thing entirely to see such verbose output from free models about, well, me . The first two, I've made peace with (nothing I can do about it anyway). The last one picks quite fiercely at old trauma that really makes me reconsider my socials in general, not just HN. But mayb…

From another perspective, it's like hearing others judging you behind your back. First few times it's awkward and maybe even annoying, but given enough time you stops to give a damn about it. But, the problem is real if it's a nation states or megacorps are doing it. They'll use such tech in an unjustified way, make a misjudgement, and then ask you to explain yourself out of the situation. Yeah, they're definitely go…

I have a sneaking suspicion this is going to be 21st century communism's (read China's) fatal flaw: the corrosive effect of panopticon monitoring on population productivity.

Because eventually apparatchiks with the data at their fingertips are going to use it to rule out the next Einsteins from participating in {insert major Chinese project}, and you've effectively self-selected at scale for people whose shared characteristic is "not being different."

We'll see if the US and Europe course correct on individual freedom enough to reap the benefits of that though.

Re: Profiling Hacker News users based on their comments

#86

Post by ethbr1 »

simonw wrote: post #35
xnx wrote: post #34

Earlier quoted context omitted.

"Make an image of a kākāpō parrot riding a bicycle"

I usually ask for a skateboard, kākāpō have dumpy little legs.

{Decades in the future} 'Simon Willison's most enduring contributions to modern computing are the pervasive edge case optimizations present in all modern AIs to draw pictures of animals with inconvenient limb arrangements using a variety of human motive devices.'

Re: Profiling Hacker News users based on their comments

#87

Post by inetknght »

zoogeny wrote: post #60

It is interesting that Marc Andreesen was having a bit of a X crash out over his belief that introspection is bad [1] I disagree because I tend to seek a middle way. I would agree that too much (excessive) introspection is bad. But I would argue that too little is equally bad. I think obsessively examining ones own comment history would verge on excessive. I'm wondering how much LLM analysis of my public and private…

> I think obsessively examining ones own comment history would verge on excessive

If you write a journal, do you not sometimes review the things you've written?

This is no different.

If you read the whole journal every day before writing a new entry, then perhaps it is a but excessive. But a review once or twice a year is not a bad thing.

Re: Profiling Hacker News users based on their comments

#88

Post by martin-t »

alexpotato wrote: post #73
martin-t wrote: post #66

Earlier quoted context omitted.

I thought I had deja vu when reading your comment so I searched and found that you wrote something very similar 6 years ago, then 4 months ago and then 3 comments within the last month. Out of curiosity and without meaning it to sound like an accusation, did you write such similar posts by hand or do you use some form of automation for commenting?

It’s funny because someone asked me about this on Twitter too. Specifically, how was I able to reply to tweets of other people with a relevant Twitter thread I had already written. It’s all manual and I guess just how my brain works. My wife actually calls it “the database” because I can quickly access stories and I apparently tell them in a very similar way. I’m just as impressed that you noticed and had the Déjà vu…

Thanks for the reply.

> reply to tweets of other people with a relevant Twitter thread I had already written

I noticed I also have topics I talk about regularly and this would be really nice. Some things only need one high-effort explanation and then linking to that.

Using LLMs would in theory save some effort by rephrasing to it doesn't look copy pasted but I am strongly opposed to the mild reality distortion they are prone to doing (like hallucinating random tidbits which never happened, using "A big part of this" as a rhetorical device instead of an actual quantity, etc.) in addition to flat out lying and other mis-generation (I don't call it hallucinations since this is normal operation, not something exceptional).

Re: Profiling Hacker News users based on their comments

#89

Post by ludicrousdispla »

stego-tech wrote: post #40

This is...disquieting. It's one thing to know that it's possible, another thing to know nation states or large megacorps are doing it, but another thing entirely to see such verbose output from free models about, well, me . The first two, I've made peace with (nothing I can do about it anyway). The last one picks quite fiercely at old trauma that really makes me reconsider my socials in general, not just HN. But mayb…

as an exercise, you could simply repost the last 1000 of simonw's comments as your own

Re: Profiling Hacker News users based on their comments

#90

Post by zoogeny »

inetknght wrote: post #87
zoogeny wrote: post #60

It is interesting that Marc Andreesen was having a bit of a X crash out over his belief that introspection is bad [1] I disagree because I tend to seek a middle way. I would agree that too much (excessive) introspection is bad. But I would argue that too little is equally bad. I think obsessively examining ones own comment history would verge on excessive. I'm wondering how much LLM analysis of my public and private…

> I think obsessively examining ones own comment history would verge on excessive If you write a journal, do you not sometimes review the things you've written? This is no different. If you read the whole journal every day before writing a new entry, then perhaps it is a but excessive. But a review once or twice a year is not a bad thing.

I very much take your point, and literally the paragraph before I said "... I tend to seek a middle way". So if you think reviewing once or twice a year is right for you, then go for it.

But to answer your specific question: I do keep journals and I almost never review them. I have a box with 20+ years of handwritten journals in it. Once every couple of years I'll even open the box and flip through a few pages of a few of the books and read a couple of pages. But that is generally all I do and I have no desire to do any more than that.

I also have maybe 100 journal entries in Google Docs and to the best of my recollection I have never read a single one of them after they were written.

Lately I've been using LLMs as a kind of active journal, where I engage in a dialogue as a means to externalizing my thoughts and refining ideas through critical feedback. I have never gone back and re-read any of those conversations.

I'm not making a value claim here, just answering you honestly. It isn't that I think one should or shouldn't review their journals, it is that it doesn't give me pleasure to do so and I find no personal value in it.

Post reply on HN