Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow – A system for deep learning and differentiable computing

tensorflow.org

Post reply on HN
Print view View original

31–40 of 142 posts

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#31

Post by turbinerneiter »

6d65 wrote: post #14

Last time I looked the automatic differentiation was in a compiler branch with no immediate plans to merge in master. But overall it is promising. I even installed Swift on Linux to play with it, didn't get to ML as I have an AMD GPU and this is a can of worms. Hope it's finished one day. I would prefer for Julia ml libraries to become mainstream. But, it is what it is. Also, the ideal for me would be Rust for tensor…

I recently spend some days playing with differentiable programming in Swift on Linux:

* as you said, auto-diff is in a branch or the Google fork of the project * the only pre-built images are for Ubuntu 18.04 * on Linux, the REPL seems to somewhat broken * many libraries are assuming OSX or iOS

I don't feel a lot of hope for adoption of Swift on Linux. Apple obviously is not working on that (fair, they have no reason to do so) and the Swift community also has no focus on Linux, since ... they are in the Apple ecosystem. Meanwhile, the Open Source community is much more interested in Rust than Swift.

For the differentiable programming - this is what got me excited, but after trying, I was a bit underwhelmed. Not that it isn't great technology, its just not figured out yet. I tried to come up with a use case outside of ML and the one I tried wasn't really applicable.

I do feel however that someone will come up with something and that it will have quite some impact.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#32

Post by cs702 »

6d65 wrote: post #14

Last time I looked the automatic differentiation was in a compiler branch with no immediate plans to merge in master. But overall it is promising. I even installed Swift on Linux to play with it, didn't get to ML as I have an AMD GPU and this is a can of worms. Hope it's finished one day. I would prefer for Julia ml libraries to become mainstream. But, it is what it is. Also, the ideal for me would be Rust for tensor…

A Rust for TensorFlow (and/or a "RustTorch") would be awesome.

I hope all the work being done on improving incremental compilation[a] and developing interactive Rust REPLs like evcxr[b] makes using Rust for AI a practical reality.

[a] https://doc.rust-lang.org/edition-guide/rust-2018/the-compil...

[b] https://github.com/google/evcxr

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#33

Post by marcinzm »

pjmlp wrote: post #25
marcinzm wrote: post #20

Earlier quoted context omitted.

They presumably wanted a semi-popular statically typed language as the gains of Julia over Python aren't enough to be worth it (and Julia isn't popular enough).

On HN circles maybe. https://juliacomputing.com/case-studies/

There's a difference between popular and "not a toy language." I'm not arguing Julia isn't used, I'm arguing it's not used often enough to be a merit irrespective of other reasons.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#34

Post by 6d65 »

cs702 wrote: post #32
6d65 wrote: post #14

Last time I looked the automatic differentiation was in a compiler branch with no immediate plans to merge in master. But overall it is promising. I even installed Swift on Linux to play with it, didn't get to ML as I have an AMD GPU and this is a can of worms. Hope it's finished one day. I would prefer for Julia ml libraries to become mainstream. But, it is what it is. Also, the ideal for me would be Rust for tensor…

A Rust for TensorFlow (and/or a "RustTorch") would be awesome . I hope all the work being done on improving incremental compilation[a] and developing interactive Rust REPLs like evcxr[b] makes using Rust for AI a practical reality. [a] https://doc.rust-lang.org/edition-guide/rust-2018/the-compil... [b] https://github.com/google/evcxr

I think there are official tensorflow bindings for Rust, add well as for pytorch C++ API.

But, adding auto differentiation to match the Swift for tensorflow behaviour, sounds like a serious undertaking, and I doubt is on anyone's radar.

But yeah, I've been wanting this for a while. Shoehorning Rust everywhere is the endgame.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#35

Post by pjmlp »

marcinzm wrote: post #33
pjmlp wrote: post #25

Earlier quoted context omitted.

On HN circles maybe. https://juliacomputing.com/case-studies/

There's a difference between popular and "not a toy language." I'm not arguing Julia isn't used, I'm arguing it's not used often enough to be a merit irrespective of other reasons.

Interesting given some of the renowned names using it, probably with more revenue than plenty of Rust unicorns.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#36

Post by nsonha »

jasode wrote: post #9
VHRanger wrote: post #3

Who would want to use an Apple-centric language for ML, seriously? Apple hardware is outright incompatible to the kind of hardware we use daily in machine learning workstations.

>Who would want to use an Apple-centric language for ML, seriously? Apple hardware is outright incompatible Based on how you wrote your comment, I'm guessing you may not know this S4TF is a Google initiative. Yes, Chris Lattner used to work for Apple but he was at Google Brain during the start of this project. When his team wanted to create a language where automatic differentiation and gradient descent was a 1st-cla…

For answering a rhetorical question? Yes of course that guy who came up with this also roots for swift to gain more traction outside Apple ecosystem but the reality hasn't going in that direction so far. There are plenty of cool languages out there like f#, julia, mathematica etc and the ML mob settled for python which is an average language so there is no reason to believe they are attracted to swift.

And you just ignored the hardware comment

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#37

Post by gas9S9zw3P9c »

bodono wrote: post #23

I'm not sure this is really going to take off, it seems that most people who are abandoning TF are moving to Jax or pytorch. My own experience with Jax is that it is much easier to use then TF, just an all round more pleasant experience. It would be interesting to try this, but at this point I'm not really willing to learn 'yet another deep learning framework' and the extreme anti-user problems that TF had make me lo…

I'm skeptical of JAX. It feels good right now, but when the first TF beta version came out it was very much like that too - clean, simple, minimal, and just a better version of Theano. Then the "crossing the chasm" effort started and everyone at Google wanted to be part of it, making TF the big complex mess it is today. It's a great example of Conway's Law. I'm not convinced the same won't happen to JAX as it catches on.

PyTorch has already stood the test of time and proven that its development is led by a competent team.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#38

Post by 6d65 »

turbinerneiter wrote: post #31
6d65 wrote: post #14

Last time I looked the automatic differentiation was in a compiler branch with no immediate plans to merge in master. But overall it is promising. I even installed Swift on Linux to play with it, didn't get to ML as I have an AMD GPU and this is a can of worms. Hope it's finished one day. I would prefer for Julia ml libraries to become mainstream. But, it is what it is. Also, the ideal for me would be Rust for tensor…

I recently spend some days playing with differentiable programming in Swift on Linux: * as you said, auto-diff is in a branch or the Google fork of the project * the only pre-built images are for Ubuntu 18.04 * on Linux, the REPL seems to somewhat broken * many libraries are assuming OSX or iOS I don't feel a lot of hope for adoption of Swift on Linux. Apple obviously is not working on that (fair, they have no reason…

Yep. It's difficult to find usages for autodiff, and it looks like a very niche thing to add to a language. But, it's still cool. In a language with extensible syntax(ex: proc macros), this would sit in a library.

And I think you're right, there is a low chance that this gets traction. Especially since Chris Lattner is at SiFive, they probably will have some kind ofAI accelerators, but TF is for training rather then execution. So not sure they'll find a reason to push it.

Jeremy Howard from fast ai might be able to convince people to give it a try. But without people working on it full time, the chances are not great. Especially with a compiler fork that requires constant merges/rebases. But, who knows.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#39

Post by bodono »

gas9S9zw3P9c wrote: post #37
bodono wrote: post #23

I'm not sure this is really going to take off, it seems that most people who are abandoning TF are moving to Jax or pytorch. My own experience with Jax is that it is much easier to use then TF, just an all round more pleasant experience. It would be interesting to try this, but at this point I'm not really willing to learn 'yet another deep learning framework' and the extreme anti-user problems that TF had make me lo…

I'm skeptical of JAX. It feels good right now, but when the first TF beta version came out it was very much like that too - clean, simple, minimal, and just a better version of Theano. Then the "crossing the chasm" effort started and everyone at Google wanted to be part of it, making TF the big complex mess it is today. It's a great example of Conway's Law. I'm not convinced the same won't happen to JAX as it catches…

I know where you're coming from, but TF in my opinion was very user-hostile even on arrival. I can't tell you how much hair-pulling I did over tf.conds, tf.while_loops and the whole gather / scatter paradigm for simple indexing into arrays. I really think the people working on it wanted users to write TF code in a certain, particular way and made it really difficult to use it in other ways. Just thinking back on that time still raises my blood pressure! So far Jax is much better and I'm cautiously optimistic they have learned lessons from TF.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#40

Post by dsabanin »

turbinerneiter wrote: post #31
6d65 wrote: post #14

Last time I looked the automatic differentiation was in a compiler branch with no immediate plans to merge in master. But overall it is promising. I even installed Swift on Linux to play with it, didn't get to ML as I have an AMD GPU and this is a can of worms. Hope it's finished one day. I would prefer for Julia ml libraries to become mainstream. But, it is what it is. Also, the ideal for me would be Rust for tensor…

I recently spend some days playing with differentiable programming in Swift on Linux: * as you said, auto-diff is in a branch or the Google fork of the project * the only pre-built images are for Ubuntu 18.04 * on Linux, the REPL seems to somewhat broken * many libraries are assuming OSX or iOS I don't feel a lot of hope for adoption of Swift on Linux. Apple obviously is not working on that (fair, they have no reason…

Apple I believe has a reason to work on Swift for Linux, and that reason is their considerable cloud infrastructure and various backend services.

I’m sure being able to share domain specific Swift code between client apps and backend would be pretty high on their list of wants.

Also, little clues like the way Xcode generates SwiftPM packages in a Linux-ready fashion out of the box shows that they care at least a bit.

Having a lot of interest in the programming languages, my opinion is that Swift is a damn good one. It’s very high level, supports FP deep enough, has a great type system (that is getting better with every release), great OOP support, native performance characteristics and it still lets you get to a really low level when you need it.

I also like how they took great ideas from Haskell, Scala, Smalltalk, C# and others. I code daily in Scala and Swift, previously had done Erlang, Clojure, Common LISP, TypeScript, Ruby, Python, Haskell, OCaml, Java, PHP, C, Smalltalk and some others. In this list, Swift is now almost at the top.

They need to get Higher-Kinded Types and then it’s going to win the world (just kidding, JavaScript gets to win the world, unfortunately) :)

Post reply on HN