Live data from Hacker News

Improved fraud prevention with Radar 2.0

stripe.com

21–30 of 83 posts

Re: Improved fraud prevention with Radar 2.0

#21
post #15
post #2

Engineering manager for Stripe Radar here. Today’s update has been almost a year in the making and we’re excited to help Stripe businesses fight fraud more effectively. Here's more on what's new: https://stripe.com/blog/radar-2018 I (and the entire Radar team) are on hand to answer any questions you may have!

One of the issues that I faced during my short stint building ML models for fraud detection in debit card transactions was dealing with class imbalance. I was not completely convinced that over sampling techniques or under sampling techniques would work. My initial experiments just resulted in more false positives. Just curious if you guys faced similar problems. The other point I bring about is rather rhetorical - T…

I agree that the lack of standards and baselines in the fraud detection space isn't ideal. One example: some fraud products will build models using human labels as the target to be predicted. Radar, on the other hand, tries to predict whether a charge actually turns out to be fraudulent (we use dispute/chargeback data we get directly from card issuers/networks). These are in fact different problems and the fact that the industry generally doesn't have a consistent target makes discourse and comparisons more muddled.

(And on class imbalance: we spent quite a bit of time experimenting/analyzing how to deal with it—we found that sampling rate has a marginal impact on performance but not a huge one.)

Re: Improved fraud prevention with Radar 2.0

#22

Ok some really dumb questions if you don't mind, but how "fraud detection" works has always been one of those areas I am interested in, but not enough to seek out a practitioner and pin them down - until now ! - Any idea what the total fraud vs genuine transactions ratio is? And how that breaks down across industries? I am assuming that SaaS services don't get as much of this - i mean would people buy bingo cards wit…

>how does fraud get monetised? Once i have downloaded my millions of credit card numbers from Tor (or stolen my friends mothers wallet) I need to persuade a merchant to deliver me something - but it's always bugged me that they actually have to deliver it. to a physical address. that can presumably be traced. It all seems very low level

Easiest way is to ship stuff to some house, then grab the package off the front porch. Best if the house isn't actually inhabited. Yeah, there's some evidence left behind, but it's usually a dead-end.

Re: Improved fraud prevention with Radar 2.0

#24
I really hope that this improves the false-positive rate, as mentioned in another comment.

We've been hurt badly as a startup breaking into the US market and getting many of our genuine charges blocked by Radar (and at a "highest risk" level where it is not possible to disable rules).

As a developer, I had the best possible impression of Stripe, as they provide easily the cleanest API and best documentation of any payment provider I've used.

From a business side, it was highly frustrating losing so many of our early US sales. A lot of this was due to US banks blocking the payment, but Stripe did not handle these cases well in our case.

* Error messages presented to the customer are often ambiguous or misleading, and if you're using checkout.js you cannot customize these easily. In some cases, the customer gets a client-side "Card is declined" error, with no communication passed through to the merchant at all, which makes telephone support very difficult.

* Stripe will not provide any kind of telephone or chat support to merchants. By the time a specific blocked payment can be escalated to a "fraud specialist" the customer is usually long gone.

* Stripe will not allow you disable certain Radar rules.

* If a user tries to pay several times (due to their bank blocking the payment as suspicious), and then phones their bank to clear the payment, Stripe will often then block the payment due to "repeated attempted uses of the card". This is highly frustrating as even very determined customers who try several times and then contact their bank to resolve the issue there still get blocked by Stripe, and usually give up at that point.

* We had one payment blocked by Radar due to it being from a "high risk location" (Pakistan if I remember correctly). This represented unacceptable levels of discrimination for us. Machine learning and probability is useful, but ethically it's hugely problematic to deny people service based on their country, race, gender, age, or other attributes that they do not have control over.

My early experiences with PayPal were nothing short of terrible, but BrainTree is looking like a more and more attractive alternative to Stripe, especially with the PayPal integration built in -- if people have issues with their Credit Card they can simply pay with PayPal credit instead.

Re: Improved fraud prevention with Radar 2.0

#25

Ok some really dumb questions if you don't mind, but how "fraud detection" works has always been one of those areas I am interested in, but not enough to seek out a practitioner and pin them down - until now ! - Any idea what the total fraud vs genuine transactions ratio is? And how that breaks down across industries? I am assuming that SaaS services don't get as much of this - i mean would people buy bingo cards wit…

> how much do the "obvious" checks help - highly unlikely purchases (3 iphones) timing or physical activity (I probably won't buy books on amazon, clothes on boohoo and petrol in a garage in the same five minutes) versus the more ML / secret squirrel stuff?

ML solutions very often are just learning to codify these 'obvious' scenarios, and as a bonus sometimes less obvious ones.

You could sit in meetings for hours/days listing all the cases the fraud detection should catch, and you'll still end up missing lots. If you're stripe, you have tons of data about fraudulent purchases that you can use to learn and codify these scenarios. Importantly, you can learn which scenarios are most prevalent in your system in particular, fraud at Stripe is very likely to look different than fraud at (say) Wells Fargo.

Re: Improved fraud prevention with Radar 2.0

#26

I wish they will be a way to minimize false positives. It’s a biggger issue to block legitimate customers because they are not from the US.

For any fraud detection scheme (or any binary classification scheme, really) there’s a tradeoff between false positives and false negatives. The Radar 2.0 updates—particularly Radar for Fraud Teams—will help here in a few ways:

- With Radar for Fraud Teams, you can customize the threshold at which Radar blocks charges—so if false positives are very costly for your business (because you have large margins, e.g.), you can tune Radar to reflect that,

- Radar 2.0’s custom machine-learning models (for businesses that have enough data with Stripe) should adapt to the unique circumstances/patterns/trends of your business, and

- Radar 2.0’s ML overall has substantially improved performance, which you should see after you’ve upgraded.

Re: Improved fraud prevention with Radar 2.0

#27

Ok some really dumb questions if you don't mind, but how "fraud detection" works has always been one of those areas I am interested in, but not enough to seek out a practitioner and pin them down - until now ! - Any idea what the total fraud vs genuine transactions ratio is? And how that breaks down across industries? I am assuming that SaaS services don't get as much of this - i mean would people buy bingo cards wit…

>how does fraud get monetised? Once i have downloaded my millions of credit card numbers from Tor (or stolen my friends mothers wallet) I need to persuade a merchant to deliver me something - but it's always bugged me that they actually have to deliver it. to a physical address. that can presumably be traced. It all seems very low level Easiest way is to ship stuff to some house, then grab the package off the front p…

The easiest thing would be to buy some bitcoin.

Re: Improved fraud prevention with Radar 2.0

#28

Ok some really dumb questions if you don't mind, but how "fraud detection" works has always been one of those areas I am interested in, but not enough to seek out a practitioner and pin them down - until now ! - Any idea what the total fraud vs genuine transactions ratio is? And how that breaks down across industries? I am assuming that SaaS services don't get as much of this - i mean would people buy bingo cards wit…

We've found that fraud vs. genuine closely matches the percentage of assholes and psychopaths in the general population, somewhere ~1% to 2%. And just like in meat-space these people are the cause for increased prices and frustrating checkout experiences - i.e. payment holds and refusals.

The problem is said people can do so much damage to a profit margin as they tend to hit an online store hard and quickly, racking up huge potential reversal costs. For us, being in a specialized digital space makes this especially painful as the digital items can be "used" and no longer resell-able or recoverable, so inventory and real dollar value is lost.

Visa / MC has no real incentive to stop the fraud as the merchant in most cases is liable for the reversal - Visa / MC is just a facilitator that bends towards keeping the buyer happy (same with PayPal). 3DSecure was introduced over a decade ago to alleviate some fraud based on unauthorized \ unknown but uptake has been anemic due to poor buyer experience.

As for converting stolen card to profit, purchasing and delivering high ticket physical items online and reselling is one known method.

Re: Improved fraud prevention with Radar 2.0

#30
post #19

Earlier quoted context omitted.

Are you allowed to share what tech stack you use? e.g. TensorFlow, Spark or ...

Most of our ML stack has been developed internally given the unique constraints we have for Radar. Among other things, we need to be able to - compute a huge number of features, many of which are quite complex (involving data collected from throughout the payment process), in real-time: e.g. how many distinct IP addresses have we seen this card from over its entire history on Stripe, how many distinct cards have we s…

Broadly speaking, what approach do you use to "build simpler 'explanation models'" from the more complicated "core fraud models"? Do you learn the models separately over the training data, or does the more complicated model somehow influence the training of the simpler model?
Post reply on HN