I love the included legal timeline. Strong work!
Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
21–30 of 76 posts
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#22This project might be entirely well-intentioned, but some possibilities to be careful of, with this kind of effort:
* Intent is to sell the data, or otherwise "monetize" it, in the techbro sense.
* "Shell" effort of a specific company that wants the data.
* Shell effort of an organized crime group.
* Shell effort of a foreign intelligence agency, or terrorist group.
Awhile ago, there was a different project, which had the effect of making different US records, which were already reasonably accessible to US citizens and journalists, easily available to foreign adversaries, such as for espionage profiling and blackmail. When that project was promoted on HN, I caught the promoter seeming to use a sockpuppet account in the comments (accidentally using the wrong account to respond to themself), which I found additionally suspicious.
Even when a project is fully honest and with good intentions, we also have to consider the risks of likely other consumers of the data, which include all the possibilities above.
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#23As a veteran, I am aghast that this exists. What an invasion of people’s lives. To me this is far worse than when I was notified that a foreign adversary had stolen my military records, because at least they’re not publishing it on the web for all to see. Veterans aren’t politicians, and they don’t deserve to have their lives put on display like this without their permission. Some vets signed up because they wanted t…
I’m also a veteran and immediately had the same thought. Thank you for thoughtfully replying, I probably would’ve been more aggressively toned had I not read your reply first.
same here, was having a hard time finding appropriate words.
i guess serving our country came with some fine print that we never got the chance to read.
but how could a vietnam war veteran possibly consent to having their records accessible to the entire world via the internet?
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#24Neat, Submitted a request for my grandfather's records. Some comments: 1) May want to auto-magically handle input for things like apostrophes. E.g. "O'Hare"... It looks like somewhere in the process this data was not preserved/saved/sent, but people will probably try to search with it. Might also want to handle the accent marks and what not too 2) The terms & conditions for Step 3, the checkbox at the bottom doesn't…
> 3) I appreciate the warning on the terms and conditions about seeing things you might not want to see.
I'd echo this. I found that to be exceptionally well-written and helped me understand the records I'd receive were unlikely to be the records I was interested in, so I cancelled at that point.
Your abandon rate at that step could make for interesting reading!
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#25Spent a few minutes trying to find an answer to "why do this?" Beyond just implying that it should be done and the most I was able to find was one sentence buried amongst paragraphs and paragraphs of "what" and "how". > these materials were largely unknown and inaccessible to historians, journalists, and genealogists I think it would be worthwhile to lead with that and include a little more detail too. If there isn't…
I think it’s critically important to mention that the VA provided all this data to Ancestry.com years ago. According to the newletter op linked, Ancestry.com charges $300/year for access to this data. This unfairness is what prompted the lawsuit and ultimate release of data.
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#26Neat, Submitted a request for my grandfather's records. Some comments: 1) May want to auto-magically handle input for things like apostrophes. E.g. "O'Hare"... It looks like somewhere in the process this data was not preserved/saved/sent, but people will probably try to search with it. Might also want to handle the accent marks and what not too 2) The terms & conditions for Step 3, the checkbox at the bottom doesn't…
Thanks. The original data set, as provided by the VA, has all sorts of data errors and oddities in it. The major ones involving surnames include the inconsistent use of apostrophes in names like O’BRIEN, often written as O BRIEN, and/or vice versa — or the inconsistent formatting of MC and MAC names like MCMAHON as MC MAHON, and/or vice versa. There are also some names where the VA includes an errant dash, not meant to be a hyphen, and other mistakes, as well.
So we try our best to help a user find the veteran even with the dirty data we have. For example, there is code here (using a common NPM package) to convert a user’s potential typed accent marks to a non-accented version of the same letter. In compound surnames we will also break up the surname on a space or a hyphen and search both parts, but not if a surname part is three letters or fewer. It’s imperfect but we have to work with the data we’ve got and can’t and shouldn’t normalize or clean the underlying file.
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#27As a veteran, I am aghast that this exists. What an invasion of people’s lives. To me this is far worse than when I was notified that a foreign adversary had stolen my military records, because at least they’re not publishing it on the web for all to see. Veterans aren’t politicians, and they don’t deserve to have their lives put on display like this without their permission. Some vets signed up because they wanted t…
As a person, not a Veteran, I totally agree. This data should not be public domain. Definitely only accessible to family members (if at all). Having obtained my father’s military records, I can definitely say that I’m glad these weren’t online, searchable. Via those records I learnt much concerning my father and I’m glad I obtained them. To get those records, I had to prove I was his son, that my father was indeed de…
The military uses the gun of the taxman to force us to pay for them. If they're going to put me in a tiny cage for not paying up while they're galavanting around the world drone striking brown people and us citizens the least I can get out of it is the records.
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#28I believe you should work to limit exposure of sensitive information like SSN: while it's ok to allow search by an exact SSN, you should probably not display it unless the requestor already knows what it is. OTOH, if you have really succesfully worked to make this database public domain and do publish it somewhere (and you did, as I can see at https://archive.org/details/BIRLS_database ), this wouldn't be of much hel…
The VA worked to confirm that everyone in this dataset is deceased, in order to satisfy the judge’s order, and produced an internal document about how they did it — which we then FOIAed and posted online too. (It’s up on the site, next to the legal paperwork.) The veterans and their SSNs are believed to have been deceased prior to mid-2020, checked by the VA’s internal datasets as well as public data sets such as the SSDMF. And SSNs of deceased people are *not private*, since they are never reused. The Social Security Administration also makes copies of all deceased peoples’ original SS-5 applications available to the public under FOIA.
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#29I previously tried getting military records for my deceased grandfather. From what I can recall, it was complicated by the fact that I wasn't "next of kin", which would limit the data I'd have access to. Even my parent, who was next of kin, would have needed additional paperwork as proof, e.g. a death/marriage certificate for my grandmother. On one hand, if this works then I'll be happy to have the information I othe…
The processes for getting these very particular records (C-Files, as opposed to something like an OMPF or other better known military records) has been horrendously broken for years. They were almost completely inaccessible from this specific agency (the VBA, inside the VA) their entire existence. Only 5% of the files have been turned over to NARA, even for records that are very old.
And even now, the “processes” to get the records, as defined by a 58+-year-old law (FOIA) are not really being followed. An agency refusing to process any FOIA requests except by fax (!) is insane, in this day and age. But more specifically, it’s against the law. A letter AND an e-mail are supposed to work. Hence our use of a fax API on this website…
Furthermore, the “requirement” that a FOIA requester must hand-sign the paperwork is absolutely made up by this agency. Hence our signature widget on this website…
Point being, if they’re going to shamelessly ignore or misinterpret the federal law, we are going to just jump through those hoops and say no, we want the files, please do your jobs.
Re: Show HN: New search engine and free-FOIA-by-fax-via-web for US veteran records
#30Are you aware that the API appears to be publishing the SSN of each individual? Although I’m aware most SSNs are leaked in one breach or another, I still thought it was customary in the U.S. to attempt to keep such information somewhat protected.
Yes, that’s on purpose. SSNs of *deceased* people are public, not private. They are never reused. They are available under FOIA from other sources as well, such as the SSDMF.