Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inside the "3 billion people" national public data breach

troyhunt.com

Post reply on HN
Print view View original

381–390 of 472 posts

Re: Inside the "3 billion people" national public data breach

#381

Post by kelnos »

kaiwenwang wrote: post #378
diggan wrote: post #369

Earlier quoted context omitted.

That's not even that big? `cat big_file | grep -v my_term` would go line-by-line and show any lines matching your query. If you're doing a lot of queries, you'd probably want to index it, so you throw it into a sqlite database with the usual SQL utils. Edit: I missed you said Windows. Probably Powershell have similar utilities, so you can do `ReadFileLineByLine \r \d big_file | ReturnHitBySearchTerm \v \t \s my_term`…

Using sift on a 100GB txt file still takes multiple minutes. I haven't tried ag, but grep is supposedly slower.

If the desire is just to grep for your name, email address, whatever, and then throw the rest of the data away, I don't think waiting multiple minutes is a big deal.

Re: Inside the "3 billion people" national public data breach

#382

Post by acdha »

AnthonyMouse wrote: post #306
toomuchtodo wrote: post #53

Earlier quoted context omitted.

> If we were going to do something, we’d make government ID include an NFC token for PKI purposes since public keys can’t be compromised in the same way, but nobody is jumping to pay for that, especially in a country where you have so many people prone to wild conspiracy theories (I am especially amazed by the guys who freak about a national ID as big brother but never say a word about the credit reporting industry)…

I still don't get why people are calling these "religious fears". The parable from the book is because the problem is very old, but the problem is exactly the same as it ever was: If a central authority gives everyone a serial number then it will be used to track them by powerful institutions, which is a tool of oppression. This is the massive mistake we made with social security numbers, and their inherent insecurit…

> I still don't get why people are calling these "religious fears

That’s what the people making those claims are talking about. If you haven’t talked with paranoid religious extremists before, it’s eye-opening: they are literally saying that a mandatory government ID will serve the beast mentioned in Revelations.

That’s not the only concern or group raising it by any means but I mentioned it because governments have to consider edge cases - if you make SSN a required field you have to figure out how to avoid turning away children from those households. If you’re building a website to sell t-shirts, that’s fine but if its government services you might be breaking the law and especially might be harming people who need help (a 17 year old who ran away from that house might have trouble getting the ID they need to live independently).

> a central authority gives everyone a serial number then it will be used to track them by powerful institutions, which is a tool of oppression.

It’s only a tool of oppression if you have a government prone to abuse and without constraints. If that’s true, since the computer age the distinction increasingly useless. The Stasi paid clerks to move paper around and if you’re comparing IDs by hand having a single number is a huge timesaver. In 2024, however, all not having one means is that they use software to link them – the context for this story is the huge industry doing that for all kinds of data, and they don’t mind having to link a couple of different identifiers. Faced with an oppressive government, we should be calling for legal restrictions and accountability for leaders. Not having a unique identification number is like wearing a breastplate into battle after the invention of the machine gun.

Re: Inside the "3 billion people" national public data breach

#383

Post by wilkystyle »

slg wrote: post #374
dingnuts wrote: post #368

Earlier quoted context omitted.

isn't it funny how no government service is ever at fault, it's always just a problem of funding? The IRS is good, just under funded. Public schools are good, just under funded. The NHS is good, just under funded. The roads are good, just under funded except then funding is raised, and it's still a problem of funding. and inevitably, it's the evil side of the government (you know the one) that is to blame, even if th…

This is neither a problem of funding or any government service being at fault. This is the fault of American culture. A national ID system sounds too scary to too many Americans. Politicians aren't going to waste their political capital on pushing through something so unpopular. It really isn't any more complicated than that. There is a huge desire for some sort of national ID system and SSNs are the closest we got s…

I'm not trying to be combative, but this sentiment just doesn't pass the smell test to me.

Yes, I agree that there is a cultural undercurrent of fear around a national ID system, and I also agree that politicians are likely to game their political capital for the greatest return in their career.

What I do NOT believe is that the Social Security number just sort of came about and started being used by government services such as the IRS without anyone being responsible for that huge organizational decision or the initial (current?) lack of security controls around its implementation.

To me, it seems to be an almost certainty that it is both an organizational problem at the government service level AND (as a result) a funding problem.

Re: Inside the "3 billion people" national public data breach

#384

Post by 6031769 »

diggan wrote: post #369
Vicinity9635 wrote: post #363

Earlier quoted context omitted.

I dug into this a little and one of the files is 164GB. How do you even work with these files? That is, how would I search for my SSN on my windows box?

That's not even that big? `cat big_file | grep -v my_term` would go line-by-line and show any lines matching your query. If you're doing a lot of queries, you'd probably want to index it, so you throw it into a sqlite database with the usual SQL utils. Edit: I missed you said Windows. Probably Powershell have similar utilities, so you can do `ReadFileLineByLine \r \d big_file | ReturnHitBySearchTerm \v \t \s my_term`…

You absolutely do not want to use "-v" with that grep.

Nor do you want to use cat (UUoCA) but that's very much a minor point in comparison.

Re: Inside the "3 billion people" national public data breach

#385

Post by [deleted] »

makmanalp wrote: post #351
ses1984 wrote: post #342

Earlier quoted context omitted.

If the salt is public, what’s the point, then you can get all the salts, and combine them with every possible ssn, and you’re back where you were before.

No, that is kind of the point of a salt is that it doesn't need to be hidden - it's designed for a scenario where e.g. your database is hacked and they're visible as plaintext: https://en.wikipedia.org/wiki/Salt_(cryptography) Since the salts are random, unique to each SSN and long: a) you'll find no existing rainbow table that contains the correct plaintext for your SSN hash and b) each SSN now requires its own brut…

[deleted]

Re: Inside the "3 billion people" national public data breach

#386

Post by dredmorbius »

autoexec wrote: post #117
Eji1700 wrote: post #96

Earlier quoted context omitted.

Well then you're up against the wall of digital verification. I know there's a fuck load of situations where the banks are 100% screwing the customer to their benefit, but there's a legit conversation about people who give out their passwords, or claim they did, when money gets wiped out. If you meet all the requirements to identify yourself to the bank, at what point does the bank have to say "this is that person, a…

> If you meet all the requirements to identify yourself to the bank, at what point does the bank have to say "this is that person, and that transaction is legal". Our current system is entirely built on ridiculous levels of trust, mostly for convenience / cost saving reasons. I've made payments over the phone with nothing more than the information found on the bottom of every check I've ever sent. I routinely hand my…

Is that "ridiculous" as in excessively stringent or weak? Because that phrase can be read either way. From the examples you give I'm presuming the latter.

Note that payments or deposits to a given account require little authentication over the destination though more for the payee. I've long been amused by US banks which require me to authenticate to an ATM to make a payment but will accept cheques dumped into a deposit slot.

I agree that the system mostly works, but fraud costs are in the billions, and that's U.S. credit cards alone:

"As Nationwide Fraud Losses Top $10 Billion in 2023, FTC Steps Up Efforts to Protect the Public " (2024) https://www.ftc.gov/news-events/news/press-releases/2024/02/...>

The denominator is roughly $4 trillion, so it's an 0.25% fraud rate:

"The Average Number of Credit Card Transactions Per Day & Year" https://www.cardrates.com/advice/number-of-credit-card-trans...>

Re: Inside the "3 billion people" national public data breach

#387

Post by LadyCailin »

meowface wrote: post #297
acidburnNSA wrote: post #198

Earlier quoted context omitted.

It can't be reversed, unfortunately. base64 has been peer proven as mathematically unhackable.

Same for base16. That's why those pesky hash digests always use it.

rot13 is way faster though.

Re: Inside the "3 billion people" national public data breach

#388

Post by slg »

wilkystyle wrote: post #383
slg wrote: post #374

Earlier quoted context omitted.

This is neither a problem of funding or any government service being at fault. This is the fault of American culture. A national ID system sounds too scary to too many Americans. Politicians aren't going to waste their political capital on pushing through something so unpopular. It really isn't any more complicated than that. There is a huge desire for some sort of national ID system and SSNs are the closest we got s…

I'm not trying to be combative, but this sentiment just doesn't pass the smell test to me. Yes, I agree that there is a cultural undercurrent of fear around a national ID system, and I also agree that politicians are likely to game their political capital for the greatest return in their career. What I do NOT believe is that the Social Security number just sort of came about and started being used by government servi…

>What I do NOT believe is that the Social Security number just sort of came about and started being used by government services such as the IRS without anyone being responsible for that huge organizational decision or the initial (current?) lack of security controls around its implementation.

They didn’t “just sort of come about”, they were created for this exact purpose of tracking government services. Over the years, the number of government services expanded because of the lack of other alternative like I said.

And the lack of security around SSNs is because they weren't intended to be secret. It is generally private sector groups like banks and credit agencies that have turned this into a problem by treating SSNs as if they are a proof of identification. They were created as usernames, but people treated them as passwords.

Re: Inside the "3 billion people" national public data breach

#389

Post by ggm »

ruthmarx wrote: post #327
ggm wrote: post #184

Earlier quoted context omitted.

Funny you should say that. Australia is trying to launch TEx -designed on open-source models to do this kind of thing. It's hitting the usual roadblocks of public acceptance of government mandated ID, in an economy which trashed the "australia card" idea back in the 80s. We're wiser now, we've been frogs boiled slowly: the downsides of central safe ID/auth are outweighed by the risks of loss of info giving everyone 1…

> layer-2 logs on our phones are constant. Huh?

Every phone provider has a log of the IMEI binding to cell tower and triangulation over multiple towers. Call logs are one thing, carrier cell connect and disconnect is another.

If your phone is on, your position in time and space to some circular error is also known, continuously.

To say nothing of Bluetooth that's with the advertising hoardings and inside the store mainly.

Basically, any privacy nut with a phone and simcard is in denial.

Re: Inside the "3 billion people" national public data breach

#390

Post by Vicinity9635 »

diggan wrote: post #369
Vicinity9635 wrote: post #363

Earlier quoted context omitted.

I dug into this a little and one of the files is 164GB. How do you even work with these files? That is, how would I search for my SSN on my windows box?

That's not even that big? `cat big_file | grep -v my_term` would go line-by-line and show any lines matching your query. If you're doing a lot of queries, you'd probably want to index it, so you throw it into a sqlite database with the usual SQL utils. Edit: I missed you said Windows. Probably Powershell have similar utilities, so you can do `ReadFileLineByLine \r \d big_file | ReturnHitBySearchTerm \v \t \s my_term`…

[dead]
Post reply on HN