Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

300M Freely Downloadable Pwned Passwords

troyhunt.com

Post reply on HN
Print view View original

21–30 of 184 posts

Re: 300M Freely Downloadable Pwned Passwords

#21

Post by drchickensalad »

r0f1 wrote: post #19

What would be the best data structure for using this in, say, a Python script? I imagine just putting it into a dictionary (hash map etc.) won't work because of the size.

If you want it lossless, a trie can be great. Not sure if one implemented in Python would be worth it though; should be a package with it in C.

Re: 300M Freely Downloadable Pwned Passwords

#25

Post by flashdance »

shoo wrote: post #22
pwdisswordfish wrote: post #11

Earlier quoted context omitted.

http://inutile.club/estatis/password-security-checker/

worth reading the T&Cs for a chuckle

I should have read that before agreeing to it...

Well, I better get prepped to fight the Estatis Inc. Retaliatory Creature to retain full ownership of my soul. Thanks for the heads up.

Re: 300M Freely Downloadable Pwned Passwords

#26

Post by r3bl »

r_singh wrote: post #5

Is it safe to test my password on this website? (because I just did)

You're probably okay, but Troy probably has the query stored somewhere, hence, he doesn't recommend that.

It is much, much safer to download the data and search for your passwords from inside the local copy of the data, and that's exactly what I'm gonna do later tonight.

Re: 300M Freely Downloadable Pwned Passwords

#28

Post by pfg »

tomschlick wrote: post #27
j_s wrote: post #24

Would it make sense to host the file on a cheap OVH/Scaleway VPS with unlimited bandwidth? I guess CloudFlare doing it for free beats that though!

"Unlimited" as in, "as soon as you start becoming a problem we drop you".

OVH lets you use the advertised bandwidth 24/7, if you wish to do so. Naturally, this doesn't replace a CDN, and their peering isn't the best, so some routes might be congested and won't be that fast.

I have no experience with Scaleway on this, but based on what I've heard about them in the past, I imagine their policy is roughly the same.

Re: 300M Freely Downloadable Pwned Passwords

#30

Post by bradleyjg »

captn3m0 wrote: post #2

Going to generate a bloom-filter from this dataset tonight. Troy mentions some arguments against torrents, but it is better to have a authoritative torrent than none, imo.

A signed minimal perfect hash function may be a better bet. You can get down to around 2.68 bits per key plus w sign bits where the false positive rate is 2^-w.

For false positive rate of 2^-9 (0.00195) that's 447 MB, which is slightly less than an optimal bloom filter for the same number of items, and lookups will be considerably faster.

Construction time and the fact that you can't add to it without rebuilding the whole thing are the downsides. But given the application I don't think they matter much.

http://sux4j.di.unimi.it/docs/it/unimi/dsi/sux4j/mph/Minimal...

Post reply on HN