Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reproducing Hacker News writing style fingerprinting

antirez.com

Post reply on HN
Print view View original

161–168 of 168 posts

Re: Reproducing Hacker News writing style fingerprinting

#161

Post by SoftTalker »

brookst wrote: post #125
SoftTalker wrote: post #115

Earlier quoted context omitted.

Nothing wrong with implying that people ought to behave according to mainstream social norms.

Isn’t that the same as saying that counterculture, fringe culture, and subcultures ought not exist?

I wouldn't take it that far, individual rights are important but so is functioning society. Counterculture is often a cyclic/youthful reaction to things that have been established as "better" or "proper" by older generations. Most rules for life such as the 10 Commandments have sort of a "deny youself some pleasure because it's better for the group" vibe that have been learned and re-learned over many centuries.

Re: Reproducing Hacker News writing style fingerprinting

#162

Post by 6510 »

selcuka wrote: post #116
6510 wrote: post #110

Earlier quoted context omitted.

I never knew A can be like B without B being like A.

The matching score is probably the same, or very close in both ways, but this fact does not necessarily help in a three-way scenario: A B: 80% A C: 90% B C: 70% When you search for A the best match will be C, but if you start with B it will be A. If one of the accounts has a smaller sample set as in GP's case, the gap could be quite big.

I'm still in disbelief. I think one should run the operation in reverse after obtaining result set.

Re: Reproducing Hacker News writing style fingerprinting

#163

Post by dang »

milesrout wrote: post #112
LoganDark wrote: post #97

Earlier quoted context omitted.

[flagged]

[flagged]

You've crossed into personal attack here. We ban accounts that do that, and you've been doing it in other threads too—for example:

https://news.ycombinator.com/item?id=43662951

https://news.ycombinator.com/item?id=43662889

If you keep this up, we're going to have to ban you again.

If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.

Re: Reproducing Hacker News writing style fingerprinting

#164

Post by jjk7 »

milesrout wrote: post #153
heresie-dabord wrote: post #140

Earlier quoted context omitted.

> If someone [test] then he is an idiot. > Anyone [test] when [test] is truly a moron. These structures are worse habits in communication than subtle, colloquially interchangeable word choices.

This isn't a habit of communication. I honestly mean it: if you get upset that someone said you "should" do something, but you are fine when they say you "ought to" do it, then you must be stupid. They mean the same thing in modern English.

Yes but words hold memories to others. Since 'ought to' is less frequently used it doesn't 'trigger' people the same way.

Most people are emotion-first, how the words make them feel is more important than the definitions of them. Being emotion-first doesn't make them stupid.

Re: Reproducing Hacker News writing style fingerprinting

#165

Post by milesrout »

jjk7 wrote: post #164
milesrout wrote: post #153

Earlier quoted context omitted.

This isn't a habit of communication. I honestly mean it: if you get upset that someone said you "should" do something, but you are fine when they say you "ought to" do it, then you must be stupid. They mean the same thing in modern English.

Yes but words hold memories to others. Since 'ought to' is less frequently used it doesn't 'trigger' people the same way. Most people are emotion-first, how the words make them feel is more important than the definitions of them. Being emotion-first doesn't make them stupid.

Being so emotional they react wildly to one of the most common words in the English language does in fact make them stupid.

Re: Reproducing Hacker News writing style fingerprinting

#166

Post by AlexeyBelov »

scoresomefeed wrote: post #109

The original version nailed all of my accounts with terrifying accuracy. Since then I make a new account every few days or weeks. Against the rules I know. And I’ve learned a lot about HN IP tracking and funny shadowbanning-like tricks they play but dont cop to. Like I get different error messages based on the different banned ips I use. And j see different behavior and inconsistency with flagged messages (like one t…

What you're doing makes HN worse, unfortunately.

Re: Reproducing Hacker News writing style fingerprinting

#167

Post by zX41ZdbW »

antirez wrote: post #145
leetrout wrote: post #113

Earlier quoted context omitted.

My favorite which is also up to date is the ClickHouse playground. For example: SELECT * FROM hackernews_history ORDER BY time DESC LIMIT 10; https://gh-api.clickhouse.tech/play?user=play#U0VMRUNUICogRl... I subscribe to this issue to keep up with updates: https://github.com/ClickHouse/ClickHouse/issues/29693#issuec... And ofc, for those that don't know, the official API https://github.com/HackerNews/API

I didn't know there was an official API! This explains why the data is so readily available in many sources and formats. That's very cool.

With a more straightforward approach, the tool can be reproduced with just a few queries in ClickHouse.

1. Create a table with styles by authors:

    CREATE TABLE hn_styles (name String, vec Array(UInt32)) ENGINE = MergeTree ORDER BY name
2. Calculate and insert style vectors (the insert takes 27 seconds):

    INSERT INTO hn_styles WITH 128 AS vec_size,
    cityHash64(arrayJoin(tokens(lower(decodeHTMLComponent(extractTextFromHTML(text)))))) % vec_size AS n,
    arrayMap((x, i) -> i = n, range(vec_size), range(vec_size)) AS arr
    SELECT by, sumForEach(arr) FROM hackernews_history GROUP BY by
3. Find nearest authors (the query takes ~50 ms):

    SELECT name FROM hn_styles ORDER BY cosineDistance(vec, (SELECT vec FROM hn_styles WHERE name = 'antirez')) LIMIT 25

        ┌─name────────────┬─────────────────dist─┐
     1. │ antirez         │                    0 │
     2. │ geertj          │ 0.009644324175144714 │
     3. │ mrighele        │ 0.009742538810774581 │
     4. │ LukaAl          │ 0.009787061201638525 │
     5. │ adrianratnapala │ 0.010093164015005152 │
     6. │ prmph           │ 0.010097599441156513 │
     7. │ teilo           │ 0.010187607877663263 │
     8. │ lukesandberg    │  0.01035981357655602 │
     9. │ joshuak         │ 0.010421492503861374 │
    10. │ sharikous       │  0.01043547391491162 │
    11. │ lll-o-lll       │  0.01051205287096002 │
    12. │ enriquto        │ 0.010534816136353875 │
    13. │ rileymat2       │ 0.010591026237771195 │
    14. │ afiori          │ 0.010655186410089112 │
    15. │ 314             │ 0.010768594792569197 │
    16. │ superice        │ 0.010842615688153812 │
    17. │ cm2187          │  0.01105111720031593 │
    18. │ jorgeleo        │ 0.011159407590845771 │
    19. │ einhverfr       │ 0.011296755160620009 │
    20. │ goodcanadian    │ 0.011316316959489647 │
    21. │ harperlee       │ 0.011317367800365297 │
    22. │ seren           │ 0.011390119122640763 │
    23. │ abnry           │ 0.011394133096140235 │
    24. │ PraetorianGourd │ 0.011508457949426343 │
    25. │ ufo             │ 0.011538721312575051 │
        └─────────────────┴──────────────────────┘

Re: Reproducing Hacker News writing style fingerprinting

#168

Post by girvo »

Frieren wrote: post #33

It works for me. The accounts I used long time ago are there in high positions. I guess that my style is very distinctive. But I also have seen some accounts that seem to be from other non-native English speakers. They may even have a Latin language as their native one (I just read some of their comments, and, at minimum, some of them seem to also be from the EU). So, I guess, that it is also grouping people by their…

Amusingly, my accounts closest (and mutual) match is from a person from the UK, and I'm a Kiwi/Australian. Though I speak and write kind of weirdly, very UK-like in some ways, and reading through that accounts comments we really do write alike!
Post reply on HN