In my case and I suppose for most HN posters there'd be little point—for on demand Ycombinator would be compelled to hand over email and IP addresses to Government—as I'd reckon in most instances that'd be a much easier and faster way of obtaining relevant information.
In an era when privacy has become hugely diminished under the hands of both governments and corporate interests it raises the question of what rights to anonymity anyone has in either a public or private forum, and at present there's little if any consensus on this which ought to signal that any such project is premature.
Unlike yours truly—who usually speaks his mind irrespective of whether he's known to his audience or does so anonymously—many will not speak their minds out of fear of being ridiculed, or humiliated, or exposed, or out of the risk of offending—risking the breakup of a friendship, etc. Same goes for whistleblowers whose public utterances, if not done anonymously, usually costs them their jobs.
If people fear that their autonomy to act in an anonymous manner has been removed then they're unlikely to act at all, silence being the better part of discretion.
This would have huge negative repercussions for society, our institutions and our governance—after all, the secret ballot is one of the cornerstones of our democracies. If we're not careful AI could undermine the ballot by unmasking what users think or how they actually vote and it's not hard to see how this would lead to coercion thence totalitarian government.
That said, in this world of widespread almost instant communications, actors who intentionally act out of bad faith can do widespread damage, especially so when they do so anonymously. Knowing who they are would minimize the damage they are able to cause.
Similarly, in a distantly-related post on HN a few days ago I referred to the increasing loss of respect for our important institutions and for the way we're being governed and how I thought that faith could be restored. There, I suggested that as a part of that process we need to unmask the hidden processes of government and that this would also include the naming of those who originate policy, law, etc.:
"If we're to restore any faith in our governance then this protection [hiding originators of policy] must stop. Decisions made by government employees must be open to public scrutiny, similarly, the origins of government policy—laws, regulations etc.—must be traceable back to its source (those who initiated said policies).
Systems without accountability will always become corrupt."
Thus, there's a real dichotomy at work here. For some things anonymity is essential, at other times it's a curse. And from the many recent instances of where the gnomes within government haven't acted in our best interests then I'm damned sure that putting AI to work here won't bode well for us either.
I've little doubt that the technology will be abused, and by virtue of the fact it will automatically silence a large proportion of the population who need speak out and who should do so anonymously in the interests of all. Even if they aren't targeted directly just knowing that there are systems in place that have the potential to expose them would be sufficient to silence many—as AI analysis of their words could be used to determine their identity at any future time (living with ongoing stress from potential exposure of one's ID would likely be unbearable for some).
Given past history and current bad behavior of governments in these areas, I do not believe that it is possible to put such a system in place that would gain the full confidence of all players involved. It would have to have sufficient protections locked in place to provide full public accountability as well as having inbuilt mechanisms that would ensure the system could not be abused by governments. At present, such conditions cannot be realistically met—not by a long shot.
Before anyone or any entity could let AI loose on this project and simultaneously state with all honesty that sufficient protections were in place for the project to proceed with safety would require many other prerequisite protections and 'safety measures'—which currently do not exist—to be incorporated (locked) into our governance. For instance, a whole raft definitions and concomitant laws pertaining to privacy are needed—and that's just for starters.
No doubt this project will proceed without those prerequisite protections, ipso facto, it will also be abused.
PS: note my quoted point about government policy etc. being open to public scrutiny. Here such questions arise such as where did this idea originate, what are the names of its instigators and what are their motives for instigating this development—not to mention others such as what are their qualifications, experience, etc. (perhaps, given the enormous potential of this AI application to damage society, we may even need to pose questions concerning their political beliefs and allegiances).
It's no accident that this information is missing with this announcement.