Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Eclipse Deeplearning4j

projects.eclipse.org

Post reply on HN
Print view View original

21–30 of 41 posts

Re: Show HN: Eclipse Deeplearning4j

#21

Post by deepGem »

riku_iki wrote: post #17
crockpotveggies wrote: post #15

Earlier quoted context omitted.

Yes, so one of the core libraries within the DL4J project is datavec which is ETL-focused. One key problem that we discovered - and fixed - was that reading and transforming data for training could bottleneck a multi-GPU process. You spend a lot of $$$ on a deep learning computer, but making the library performant enough so that you could load data at the same rate the GPUs could consume it was challenging. This scal…

Why mix ETL and training? I am using TF, and in my workflow I first do all ETL in some separate process, dump all training/validation data into TFRecord file, and then my training program consumes it. Clear separation of concerns without any performance penalty. And I can iterate over training logic with various parameters as many times as I want without touching ETL.

But what if your training logic needs a change in ETL - You have to iterate on ETL too. So doing it inline makes a lot of sense. Microsoft has done something like this with SQL server + R offering. Personally I find the MS approach quite appalling. You have to load your model in a PLSQL script, so the ETL + running the model is seamless, but imagine debugging and oh forget about multi GPU support.

Re: Show HN: Eclipse Deeplearning4j

#22

Post by agibsonccc »

deepGem wrote: post #21
riku_iki wrote: post #17

Earlier quoted context omitted.

Why mix ETL and training? I am using TF, and in my workflow I first do all ETL in some separate process, dump all training/validation data into TFRecord file, and then my training program consumes it. Clear separation of concerns without any performance penalty. And I can iterate over training logic with various parameters as many times as I want without touching ETL.

But what if your training logic needs a change in ETL - You have to iterate on ETL too. So doing it inline makes a lot of sense. Microsoft has done something like this with SQL server + R offering. Personally I find the MS approach quite appalling. You have to load your model in a PLSQL script, so the ETL + running the model is seamless, but imagine debugging and oh forget about multi GPU support.

The part of the ETL you'd want to "crystallize" is just the transformation from raw data to feature vector.

Beyond that, you already "change in ETL" when you experiment. I'm not sure how this changes anything.

Out of nowhere you've sprinkled in "multi gpu support" which I'm not sure is relevant here. Do you mean as part of training?

We handle the gpu bits for you. All you do is define your transform logic, it runs on one of our backends like spark and then when you go to allocate a tensor boom gpu.

There's no special compilation or process needed to make this happen.

Dl4j supports multi gpu training out of the box. All you need to do is uses our parallelwrapper module.

We can also do distributed training with gpus on spark as well (yes this includes cudnn).

You've also for some reason decided to attach an open ended coding library where you can do whatever you want to a database?

Baked in ML in the database servers is already notoriously bad. The whole point of what we're doing is to provide a middle ground.

Data engineers have to do this anyways.

Re: Show HN: Eclipse Deeplearning4j

#23

Post by deepGem »

agibsonccc wrote: post #22
deepGem wrote: post #21

Earlier quoted context omitted.

But what if your training logic needs a change in ETL - You have to iterate on ETL too. So doing it inline makes a lot of sense. Microsoft has done something like this with SQL server + R offering. Personally I find the MS approach quite appalling. You have to load your model in a PLSQL script, so the ETL + running the model is seamless, but imagine debugging and oh forget about multi GPU support.

The part of the ETL you'd want to "crystallize" is just the transformation from raw data to feature vector. Beyond that, you already "change in ETL" when you experiment. I'm not sure how this changes anything. Out of nowhere you've sprinkled in "multi gpu support" which I'm not sure is relevant here. Do you mean as part of training? We handle the gpu bits for you. All you do is define your transform logic, it runs on…

Adam, sorry guess I was not clear. What I said was that the MS Sql server + R strategy of baking in ML in PLSQL is appalling and doing something like what you are doing with DL4J is perhaps the right approach. The multi GPU support rant was for SQL Server + R, not DL4J.

"You've also for some reason decided to attach an open ended coding library where you can do whatever you want to a database?" I didn't catch this part.

Re: Show HN: Eclipse Deeplearning4j

#24

Post by agibsonccc »

deepGem wrote: post #23
agibsonccc wrote: post #22

Earlier quoted context omitted.

The part of the ETL you'd want to "crystallize" is just the transformation from raw data to feature vector. Beyond that, you already "change in ETL" when you experiment. I'm not sure how this changes anything. Out of nowhere you've sprinkled in "multi gpu support" which I'm not sure is relevant here. Do you mean as part of training? We handle the gpu bits for you. All you do is define your transform logic, it runs on…

Adam, sorry guess I was not clear. What I said was that the MS Sql server + R strategy of baking in ML in PLSQL is appalling and doing something like what you are doing with DL4J is perhaps the right approach. The multi GPU support rant was for SQL Server + R, not DL4J. "You've also for some reason decided to attach an open ended coding library where you can do whatever you want to a database?" I didn't catch this pa…

Right so I think we were agreeing that how database servers bake in the ML is a bit weird and not the way to go.

The "open ended coding library" is datavec here. Comparing them here is only really semi valid. The processes there are definitely brittle.

Re: Show HN: Eclipse Deeplearning4j

#25

Post by sandGorgon »

agibsonccc wrote: post #10
sandGorgon wrote: post #8

This is super, super cool! Love the work you guys have been putting out. How do you generally see the community mindshare in space especially with https://techcrunch.com/2017/02/13/yahoo-supercharges-tensorf... (you guys are also mentioned there) You guys run this at scale, but Yahoo's TensorFlowOnSpark does look very enticing with its "Easily migrate all existing TensorFlow programs with <10 lines of code change" pu…

Doesn't matter. We don't need to be the #1 framework. We're going to be focused on model import and integrating with the big data ecosystem. None of these frameworks you're mentioning integrate properly, are really supported by an actual community, and don't get meaningful updates. The biggest problem you run in to pretty quickly is maintenance. TF and spark both update quickly. Every time someone has attempted to do…

So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly?

Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.

Re: Show HN: Eclipse Deeplearning4j

#26

Post by agibsonccc »

sandGorgon wrote: post #25
agibsonccc wrote: post #10

Earlier quoted context omitted.

Doesn't matter. We don't need to be the #1 framework. We're going to be focused on model import and integrating with the big data ecosystem. None of these frameworks you're mentioning integrate properly, are really supported by an actual community, and don't get meaningful updates. The biggest problem you run in to pretty quickly is maintenance. TF and spark both update quickly. Every time someone has attempted to do…

So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly? Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.

Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet.

Granted, a ton of it is TF. Google is doing an amazing job now.

We're trying to be a moderate middle ground.

TF and pytorch and anything in python tends to be an interface.

Core logic still operates in C/C++.

That's great when you want speed, but you end up missing the benefits of the JVM (the tooling is great for monitoring and things folks need at scale).

So we do everything via JNI where we push down the math one block with minimal overhead if any.

That and if you want to write production code, you don't need to push logic down to c. You can do something JVM based instead, which gives you kotlin,scala,clojure,..

Re: Show HN: Eclipse Deeplearning4j

#27

Post by sandGorgon »

agibsonccc wrote: post #26
sandGorgon wrote: post #25

Earlier quoted context omitted.

So just trying to understand this - does dl4j become the essential middle layer if someone wants to run Tensorflow on spark seamlessly? Because it seemed on the surface that you guys replaced tf. If you guys offer a better ecosystem where tf and spark work together (but don't really rip out tf), then it is even more awesome.

Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet. Granted, a ton of it is TF. Google is doing an amazing job now. We're trying to be a moderate middle ground. TF and pytorch and anything in python tends to be an interface. Core logic still operates in C/C++. That's great when you want speed, but you end up missing the benefits…

i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark".

Because right now, it looks like it is tensorflow vs dl4j.

Re: Show HN: Eclipse Deeplearning4j

#28

Post by agibsonccc »

sandGorgon wrote: post #27
agibsonccc wrote: post #26

Earlier quoted context omitted.

Not just TF: pytorch, keras,mxnet,.. Like I said: We won't be the #1 framework data scientists use. There's a ton of churn in that space yet. Granted, a ton of it is TF. Google is doing an amazing job now. We're trying to be a moderate middle ground. TF and pytorch and anything in python tends to be an interface. Core logic still operates in C/C++. That's great when you want speed, but you end up missing the benefits…

i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark". Because right now, it looks like it is tensorflow vs dl4j.

Honestly hard for me to care..They can both compete as well as integrate.No offense here but model import existing will become a common thing as things like ONNX become standard.

We play up what's unique about Dl4j anyways. It's right in the name. We're square focused on the JVM.

We also hedge our bets against spark. Most of our logic doesn't even run on spark.

Spark is just a facilitator..it's not where anything that matters runs. We could just as easily use flink or apex here too. Those are also JVM based streaming engines.

We can't overplay spark because most of our deployments won't even be with spark. We don't even use spark in our own production inference tools. We just deploy as a microservice.

Re: Show HN: Eclipse Deeplearning4j

#29

Post by albertzeyer »

stolsvik wrote: post #11

For anyone that hasn't checked out Deeplearning4j yet, and think that Python is the only place where the cool AI stuff happens, you really should check this project out! Their Gitter channel is absolutely kicking, where lots of the actual devs, including CTO and co-founder Adam Gibson, hang out and /really/ answer /any/ question - I find their responsiveness stunning! (I discussed and reported a minor improvement to…

If you prefer Java over Python, some of the other more popular frameworks also have Java bindings. E.g. TensorFlow or CNTK.

Re: Show HN: Eclipse Deeplearning4j

#30

Post by sandGorgon »

agibsonccc wrote: post #28
sandGorgon wrote: post #27

Earlier quoted context omitted.

i think this aspect of dl4j gets lost in the overall message. For me, this is much more powerful - "use dl4j if you want to have a seamless experience running tensorflow/keras/mxnet on spark". Because right now, it looks like it is tensorflow vs dl4j.

Honestly hard for me to care..They can both compete as well as integrate.No offense here but model import existing will become a common thing as things like ONNX become standard. We play up what's unique about Dl4j anyways. It's right in the name. We're square focused on the JVM. We also hedge our bets against spark. Most of our logic doesn't even run on spark. Spark is just a facilitator..it's not where anything tha…

Well interesting parallel you give here !

I have personally been campaigning for ONNX to merge/leverage Apache Arrow (https://news.ycombinator.com/item?id=15195658). It probably makes sense for the efforts to build on top of each other.

YMMV ;)

Post reply on HN