Using R to detect fraud at 1M transactions per second [video]
blog.revolutionanalytics.com
Using R to detect fraud at 1M transactions per second [video]
1–10 of 72 posts
Re: Using R to detect fraud at 1M transactions per second [video]
#2Does anybody use R in production services or just for exploratory work?
It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
Re: Using R to detect fraud at 1M transactions per second [video]
#3> PROS has been using R for a while in development, but found running R within SQL Server 2016 to be 100 times (not 100%, 100x!) faster for price optimization. "This really woke us up that we can use R in a production setting ... it's truly amazing," he says.
WOW if this is even half true we have a new area of R.
Re: Using R to detect fraud at 1M transactions per second [video]
#4> PROS has been using R for a while in development, but found running R within SQL Server 2016 to be 100 times (not 100%, 100x!) faster for price optimization. "This really woke us up that we can use R in a production setting ... it's truly amazing," he says. WOW if this is even half true we have a new area of R.
What does it mean to run R "within" SQL Server here?
Re: Using R to detect fraud at 1M transactions per second [video]
#5Does anybody use R in production services or just for exploratory work? It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
> It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
I wouldn't say 1% of programs in R written need that speed. I personally use it for small projects (Besides a few Spark side projects) and I am out putting Reports.
I really would like someone to show an actual example of this happening in 2016.
Re: Using R to detect fraud at 1M transactions per second [video]
#6Does anybody use R in production services or just for exploratory work? It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
Afaik Bloomberg uses it extensively for internal data visualization tools.
Re: Using R to detect fraud at 1M transactions per second [video]
#7Does anybody use R in production services or just for exploratory work? It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
> It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work. I wouldn't say 1% of programs in R written need that speed. I personally use it for small projects (Besides a few Spark side projects) and I am out putting Reports. I really would like someone to show an actual example of this happening in 2016.
I do it at my company. I prototype in R, and then end up having to rewrite chunks of it in Python so it can be worked into our application, which right now is exclusively Python.
It's not a matter of performance, it's just because it would be an enormous amount of engineering overhead to start calling R from inside the Python app
Re: Using R to detect fraud at 1M transactions per second [video]
#8> PROS has been using R for a while in development, but found running R within SQL Server 2016 to be 100 times (not 100%, 100x!) faster for price optimization. "This really woke us up that we can use R in a production setting ... it's truly amazing," he says. WOW if this is even half true we have a new area of R.
What does it mean to run R "within" SQL Server here?
SQL Server R Services https://msdn.microsoft.com/en-us/library/mt604845.aspx
New for SQL Server 2016
Re: Using R to detect fraud at 1M transactions per second [video]
#9Does anybody use R in production services or just for exploratory work? It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
Part of that may stem from R and most (all?) of its libraries being licensed under GPL.
Re: Using R to detect fraud at 1M transactions per second [video]
#10> PROS has been using R for a while in development, but found running R within SQL Server 2016 to be 100 times (not 100%, 100x!) faster for price optimization. "This really woke us up that we can use R in a production setting ... it's truly amazing," he says. WOW if this is even half true we have a new area of R.
That statement is overly vague and sounds like marketing BS.
I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the data into and out of the R session. We started out using disk based I/O and ended up using using rJava so our streaming system could communicate with R. In that case we did get a 100X speed up between our first iteration and the final version which used rJava to serialize the data.
So basically, the major bottleneck was not R. It was the communication with R. In the article R is installed on the same hardware as SQL Server, which should automatically give it a speedup with streaming data.
If Microsoft also has an optimized way to get data from SQL Server to R I can see how they got a 100X speedup. In certain cases using the MKL libraries can give you that as well from faster scoring, but I suspect the speedup just comes from improving the data transfer method.