Live data from Hacker News

'Lavender': The AI machine directing Israel's bombing in Gaza

972mag.com

711–720 of 1001 posts

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#711
post #682

> One source stated that human personnel often served only as a “rubber stamp” for the machine’s decisions, adding that, normally, they would personally devote only about “20 seconds” to each target before authorizing a bombing [...] Brings the Ironies of Automation paper to mind: https://en.m.wikipedia.org/wiki/Ironies_of_Automation Specifically: If _most_ of a task is automated, human oversight becomes near useless…

Nothing a few rounds of war crimes trials, complete with gallows, in the Hague won't sort out, especially if the court decided (oh I can dream) that the execs and major investors of the company that implemented the system are culpable.

> war crimes trials

What is the alleged war crime in using a machine for target filtration?

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#712

Earlier quoted context omitted.

"I'm sure that's what was intended" Intended by who? You don't kill 13,000 children by accident.

[flagged]

The median age in Gaza in 2014 was 18. According to researchers in the US, 50%-61% of buildings have been damaged or destroyed. If Israel didn't kill tens of thousands of children it would be a statistical miracle.

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#713
post #570

Earlier quoted context omitted.

It's also interesting (and I guess typical for end-users of software) how quickly and easily something like this goes from "Here's a tool you can use as an information input when deciding who to target" to "I dunno, computer says these are the people we need to kill, let's get to it". In the Guardian article, an IDF spokesperson says it exists and is only used as the former, and I'm sure that's what was intended and…

[flagged]

Some have. Most of the IDFs current critics haven't. They get the news and conclude reasonably that the IDF deserves criticism over how it's conducting itself.

For example: "The army also decided during the first weeks of the war that, for every junior Hamas operative that Lavender marked, it was permissible to kill up to 15 or 20 civilians"

When a computer generates a target list of thousands, that's how tens of thousands of innocent people die.

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#714

Earlier quoted context omitted.

This is obviously veering way off course of the topic of AI at this point, but I imagine the residents of kibbutz be'eri and the 100+ hostages still held in Gaza would disagree that Isreal is fighting for "basically no reason." I'm interested in analysis and criticism of Israel's use of AI in this case but suggesting Israel has no causus belli is absurd.

OP didn't say Israel is fighting for basically no reason. You're twisting their words, I'll assume out of a misreading. Read the comment again. They clearly said that there's no good reason to bomb Gaza the way that they have been doing, resulting in the murder of thousands and thousands and thousands and thousands of civilians.

There is a great reason for doing it: nothing else has worked.

Unilateral withdrawal from Gaza led to the tripling of rocket attacks. Multiple peace offers have been rejected. Limited Israeli retaliation and extensive international aid has meant the Palestinian civilian population is sufficiently insulated from the violence that they have no incentive to demand peace from their leaders.

When Palestine was sending children as suicide bombers, Israel decided to play defense and built walls that dramatically lowered the efficacy of suicide bombings. So the Palestinians switched to rockets. So Israel again played defense and built the iron dome. So Hamas switched to Oct. 7th. Do you think they should play defense again?

Tell me: what peace offer do you think Israel could make the Palestinians that would lead to a lasting peace? Tell me: if Israel surrendered unconditionally to the Palestinians, would the Israelis live in peace?

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#715

As someone working in the AI field, I find this use of AI truly terrifying. Today it may be used to target Hamas and accept a relatively large number of civilian deaths as permissible collateral damage, but nothing guarantees that it won't be exported and used somewhere else. On top of that, I don't think anything is done to alleviate biases in the data (if you're used to target people from a certain group then your…

Given we don’t know what it’s using to identify people we don’t really know any biases. “Holding a military weapon” probably doesn’t contain a whole lot of bias (of course there is misidentification).

Let me quote from the article:

> Lavender learns to identify characteristics of known Hamas and PIJ operatives, whose information was fed to the machine as training data, and then to locate these same characteristics — also called “features” — among the general population, the sources explained. An individual found to have several different incriminating features will reach a high rating, and thus automatically becomes a potential target for assassination.

It literally says that they use data from known Hamas members (we don't know what this data contains) as training data which is a recipe for making biased predictions. Hamas members represent a minority in Gaza (the total population is over 2 million people) and thus the real data is heavily imbalanced[0] and unless addressed leads to bad models.

On top of that, if you know anything about Machine Learning then you should be aware of models finding spurious correlations[1] in the data that make its predictions accurate on the available training and validation data and not so much once deployed and used with real data.

[0] https://developers.google.com/machine-learning/data-prep/con...

[1] https://thegradient.pub/shortcuts-neural-networks-love-to-ch...

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#716

Years ago, scholars (such as Didier Bigo) have already raised concerns about the targeting of individuals merely based on (indirect) association with a "terrorist" or "criminal". Originally used in the context of surveillance (see Snowden revelations), such systems would target anyone who would be e.g. less than 3-steps away from an identified individual, thereby removing any sense of due process or targeted surveill…

That's not exactly a prediction. It was was standard operating procedure for Warsaw Pact nations. They used human intel which was possibly even worse because it could manipulated out of malice.

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#717
post #21

I know many people won't read past the headline, but please try to. This is the second paragraph: "In addition to talking about their use of the AI system, called Lavender, the intelligence sources claim that Israeli military officials permitted large numbers of Palestinian civilians to be killed, particularly during the early weeks and months of the conflict."

Yeah the AI is probably the most ethical part of this operation. The human leadership decided that no amount of collateral damage was too much.

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#718

Earlier quoted context omitted.

I like this point, and I do think you're rightly pointing out that the issue is that selection of targets may be done badly, not that AI specifically is in the loop. With that said, I think an important detail you're overlooking is the frictionless-ness of this process. That quote people are throwing around about something like "efficiently producing the largest volume of human targets" gets to this point pretty dire…

I agree making something bad easier is bad too. But does AI make the bad thing easier here? I see two cases here. One is that the AI has some non-negligible accuracy, and one where it doesn't. If it's somewhat accurate, then actually, using it is saving civilian lives, attacking only the active enemy. And if it's inaccurate... Then presumably whoever made it knows it, and whoever uses it knows it's merely a fig leaf…

The accuracy point is a provocative and interesting question. I'm used to it in the context of ex. medical imaging or autonomous vehicles. In the context of picking bomb targets (where even a "positive" classification is kind of ambiguous [0]) I think it's probably above my pay-grade, so I'm going to set it aside.

> whoever uses it knows it's merely a fig leaf for shooting random people

I think this is the problem, but needs a little more unpacking, because IMO it goes beyond a pure 'fig leaf'. From what I understand it's not just a way to ID who is a combatant: it actively plans bomb targets. The difference is that a fig leaf provides purely pretense, and as you point out that's nothing new: we've had automated ways of ID'ing someone as a criminal or terrorist forever. But this not only provides the pretense of ID'ing someone as a combatant, it also loads the gun and aims it for you. So to me it's more than just someone saying "oh these people were all flagged, so let's plan an attack on them", it's actually the machine drawing up the full plan and just asking you "I found combatants should I kill then [Y]/N?". Both are bad (IMO), but the second one seems like a new evolution in the automation of warfare that I find uniquely concerning.

[0] Expanding on this point a little: combatant status seems ambiguous to me because it's not really a physically measurable variable. A car crashing or an image containing a tumor are all things that can be objectively verified, but the legal worthiness of killing someone for participation in a war is a far more ambiguous concept I think. Is someone who quarters enemy troops a worthy combatant? Someone who provides logistical support? I see lots of room for ambiguity that would be ugly to encode in data.

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#719
post #525

Is there a list of congress people who support sending our tax money to Israel?

A much more concise list could be made of members of Congress that don't want to send money towards offensive weapons in Israel.

It's a vanishingly small list. Virtually everyone wearing a (D) or (R) hat is extremely interested in sending our tax money towards this purpose, which fund programs exactly like in this article.

Re: 'Lavender': The AI machine directing Israel's bombing in Gaza

#720

Earlier quoted context omitted.

> What can one do when criminals have embedded them self in the civilian population? Why don't they get out and meet the police on the battle field? We wouldn't tolerate a SWAT team blowing up a hospital if the mafia had taken over the basement, I have no idea why you think this is acceptable. > It is much better than the carpet bombing used by other nations. It is exactly like the carpet bombing used by other nation…

> We wouldn't tolerate a SWAT team blowing up a hospital if the mafia had taken over the basement, I have no idea why you think this is acceptable. While I agree with comparing Hamas to the mafia, both are criminal organizations, Hamas is more than that. It has rockets, it mascaraed civilians and holds the ideology of genociding its enemy. None of that is applicable to the mafia. So if its people are hiding in an hos…

Your argument is just whataboutism and barely worth even engaging with. It is possible that both sides of the military conflict are wrong. Being against Israel’s actions here don’t imply support for Hamas and vice versa. In this case, Israel is the one with most of the power, and they’ve used that power to kill far more people than Hamas. To defend that abuse of power is immoral.
Post reply on HN