11–20 of 72 posts
Does anybody use R in production services or just for exploratory work? It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
Afaik Bloomberg uses it extensively for internal data visualization tools.
> PROS has been using R for a while in development, but found running R within SQL Server 2016 to be 100 times (not 100%, 100x!) faster for price optimization. "This really woke us up that we can use R in a production setting ... it's truly amazing," he says. WOW if this is even half true we have a new area of R.
That statement is overly vague and sounds like marketing BS. I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the data into and out of the R session. We started out using disk based I/O and ended up using using rJava so our streaming system could communicate with R. In that case we did get a 100X speed up between our first iteration and the final version which used rJava t…
The optimized method is that you can run R inside the database in the latest version of SQL Server.
I've actually installed Windows again, just to play with this feature (though I cannot make claims to actually putting it to good use yet).
Earlier quoted context omitted.
That statement is overly vague and sounds like marketing BS. I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the data into and out of the R session. We started out using disk based I/O and ended up using using rJava so our streaming system could communicate with R. In that case we did get a 100X speed up between our first iteration and the final version which used rJava t…
> If Microsoft also has an optimized way to get data from SQL Server to R I can see how they got a 100X speedup. In certain cases using the MKL libraries can give you that as well, but I suspect the speedup just comes from improving the data transfer method. The optimized method is that you can run R inside the database in the latest version of SQL Server. I've actually installed Windows again, just to play with this…
> PROS has been using R for a while in development, but found running R within SQL Server 2016 to be 100 times (not 100%, 100x!) faster for price optimization. "This really woke us up that we can use R in a production setting ... it's truly amazing," he says. WOW if this is even half true we have a new area of R.
Earlier quoted context omitted.
That statement is overly vague and sounds like marketing BS. I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the data into and out of the R session. We started out using disk based I/O and ended up using using rJava so our streaming system could communicate with R. In that case we did get a 100X speed up between our first iteration and the final version which used rJava t…
> If Microsoft also has an optimized way to get data from SQL Server to R I can see how they got a 100X speedup. In certain cases using the MKL libraries can give you that as well, but I suspect the speedup just comes from improving the data transfer method. The optimized method is that you can run R inside the database in the latest version of SQL Server. I've actually installed Windows again, just to play with this…
"When you select this feature, extensions are installed in the database engine to support execution of R scripts, and a new service is created, the SQL Server Trusted Launchpad, to manage communications between the R runtime and the SQL Server instance."
So basically SQL Server is talking to the R session. The speedup is coming from R being installed locally and the "communication" which I've yet to figure out.
Earlier quoted context omitted.
> It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work. I wouldn't say 1% of programs in R written need that speed. I personally use it for small projects (Besides a few Spark side projects) and I am out putting Reports. I really would like someone to show an actual example of this happening in 2016.
I do it at my company. I prototype in R, and then end up having to rewrite chunks of it in Python so it can be worked into our application, which right now is exclusively Python. It's not a matter of performance, it's just because it would be an enormous amount of engineering overhead to start calling R from inside the Python app
http://blog.revolutionanalytics.com/2016/01/pipelining-r-pyt...
Also why not just switch to Pandas it really is a pretty close R clone.
Earlier quoted context omitted.
> It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work. I wouldn't say 1% of programs in R written need that speed. I personally use it for small projects (Besides a few Spark side projects) and I am out putting Reports. I really would like someone to show an actual example of this happening in 2016.
I do it at my company. I prototype in R, and then end up having to rewrite chunks of it in Python so it can be worked into our application, which right now is exclusively Python. It's not a matter of performance, it's just because it would be an enormous amount of engineering overhead to start calling R from inside the Python app
Earlier quoted context omitted.
That statement is overly vague and sounds like marketing BS. I worked on a project where we scored streaming data in R. The biggest bottleneck was getting the data into and out of the R session. We started out using disk based I/O and ended up using using rJava so our streaming system could communicate with R. In that case we did get a 100X speed up between our first iteration and the final version which used rJava t…
> If Microsoft also has an optimized way to get data from SQL Server to R I can see how they got a 100X speedup. In certain cases using the MKL libraries can give you that as well, but I suspect the speedup just comes from improving the data transfer method. The optimized method is that you can run R inside the database in the latest version of SQL Server. I've actually installed Windows again, just to play with this…
I heard the Linux SQL Server is surprisingly decent.
https://blogs.microsoft.com/blog/2016/03/07/announcing- sql-server-on-linux/
Does anybody use R in production services or just for exploratory work? It seems that once you figure out a good model in R, its almost always rewritten into either Scala or Java for real production work.
Just Say No. It'll sap your mojo. Am moving the whole thing to a blend of C, Python, and a distributed computing framework (thinking of Flink or Concord.io).
I'm assuming this wasn't real-time, real-world data (although I didn't watch the whole 1.5hr video to confirm), but the implication is that this system could process the peak load of global credit card transactions as they happened. That's pretty impressive.