Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow – A system for deep learning and differentiable computing

tensorflow.org

Post reply on HN
Print view View original

71–80 of 142 posts

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#71

Post by nestorD »

cs702 wrote: post #32
6d65 wrote: post #14

Last time I looked the automatic differentiation was in a compiler branch with no immediate plans to merge in master. But overall it is promising. I even installed Swift on Linux to play with it, didn't get to ML as I have an AMD GPU and this is a can of worms. Hope it's finished one day. I would prefer for Julia ml libraries to become mainstream. But, it is what it is. Also, the ideal for me would be Rust for tensor…

A Rust for TensorFlow (and/or a "RustTorch") would be awesome . I hope all the work being done on improving incremental compilation[a] and developing interactive Rust REPLs like evcxr[b] makes using Rust for AI a practical reality. [a] https://doc.rust-lang.org/edition-guide/rust-2018/the-compil... [b] https://github.com/google/evcxr

There are torch bindings that are used by some users but what I personaly would like is a JAX clone built on top of Rust.

I see a way to do it [0] but... I already have a PhD to finish.

[0]: a macro compiling functions into an intermediate representations that are transformed with const functions (gradient computation) at compile time and jitted into XLA at runtime.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#72

Post by [deleted] »

bartvk wrote: post #26
throw6606 wrote: post #13

Careful, folks. S4TF is pretty much dead on arrival. It was pushed aggressively by Chris Lattner (for obvious reasons) but he left Google a while ago and since then most internal users lost interest. There's nothing in Swift that's inherently suitable for ML and building the ecosystem is a ton of work; without all the political pushing, it went nowhere and is close to a "semi-abandoned research project" phase.

> since then most internal users lost interest Why do you say so?

[deleted]

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#73

Post by 6d65 »

Scipio_Afri wrote: post #50
improbable22 wrote: post #42

Earlier quoted context omitted.

> It's difficult to find usages for autodiff I guess there are lots of uses in optimisation problems, and in sampling algorithms for statistics. I don't know how easy it will be to sell Swift to people who now use Stan or Stata (or R) and don't think of themselves as programmers. > In a language with extensible syntax(ex: proc macros), this would sit in a library. And this would allow easier iteration of different de…

Does autodiff exist in R? Then why not use R since it seems it also has much ML algo support due to its focus on statistics - its used by many statisticians. Is the idea here that Swift is a more approachable language and thus this is to lower the barrier of entry to TF?

I think the main selling points of Swift to tensorflow are:

* Speed, ML learning pipelines are often bottlenecked by the data load and transformation. TF had new mechanisms (the last I've seen was TF.data). But a language compiled to native is much more flexible in that regard.

* Type safety. Sometimes issues with the models can pop long after they have been running. The hope is that typed API's will show simple errors at compile times

* Auto differentiation built into the language. If I'm not mistaken, this is more powerful than backpropagation in TF, wich also has autodiff. The idea is that this would allow for more custom models without a performance penalty. My knowledge here is limited, since it's been over 2 years since I've implemented back propagation. I've successfully forgotten most of the things I knew about ML/DL.

I don't have any experience with R, but from what I've heard, it was known to be slow. But that might have changed or I may have misunderstood the situation.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#74

Post by rudedogg »

vimy wrote: post #16
VHRanger wrote: post #3

Who would want to use an Apple-centric language for ML, seriously? Apple hardware is outright incompatible to the kind of hardware we use daily in machine learning workstations.

That might change with Apple Silicon and Apple’s new ML Compute framework.

It seems like they'll be making more consumer focused GPUs/ML chips to me. I think Apple Silicon is exciting, but I imagine for serious work you'll still need an external enclosure and a dedicated GPU from AMD.

Or Linux/Windows :(. Losing Nvidia/CUDA kind of killed ML on macOS overnight.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#75

Post by atorodius »

MiroF wrote: post #66
atorodius wrote: post #63

Earlier quoted context omitted.

This is not statically checked but it's a step in the right direction: https://pytorch.org/docs/stable/named_tensor.html

Yeah, I actually helped work on the inspo for that project https://github.com/harvardnlp/namedtensor . From what I've been able to tell, (no shade to the Pytorch team which has many different priorities) work has been somewhat slow going on the port. Further, this is dynamic type checking as you mentioned.

I see, interesting! Yeah statically checking this would be way more awesome still

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#76

Post by Razengan »

ddbb33 wrote: post #17
throw6606 wrote: post #13

Careful, folks. S4TF is pretty much dead on arrival. It was pushed aggressively by Chris Lattner (for obvious reasons) but he left Google a while ago and since then most internal users lost interest. There's nothing in Swift that's inherently suitable for ML and building the ecosystem is a ton of work; without all the political pushing, it went nowhere and is close to a "semi-abandoned research project" phase.

Can you backup your claims?

Newly created account claims something without proof and comments asking for such seem to be getting buried.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#78

Post by MiroF »

atorodius wrote: post #75
MiroF wrote: post #66

Earlier quoted context omitted.

Yeah, I actually helped work on the inspo for that project https://github.com/harvardnlp/namedtensor . From what I've been able to tell, (no shade to the Pytorch team which has many different priorities) work has been somewhat slow going on the port. Further, this is dynamic type checking as you mentioned.

I see, interesting! Yeah statically checking this would be way more awesome still

Oh I just noticed that you're one of the people behind that recent GAN compression work! Really cool stuff and a big step up this year, I've been following the field for a lil bit.

Congrats!

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#79

Post by rudedogg »

pjmlp wrote: post #22
iddan wrote: post #7

Earlier quoted context omitted.

Swift is gaining momentum outside the Apple ecosystem. Web servers are being built and the Tensorflow team explains in length it’s advantages for developing ML models with it.

What momentum? Even IBM gave up on it.

I don't have any inside knowledge, but as a developer using Swift (that followed the Swift web frameworks closely):

I think Kitura only existed as a long shot for kicking off their cloud offering. They were hoping iOS developers would code their backends in Swift/Kitura and host on IBMs cloud.

But that never happened - so Kitura was killed. Swift on the server is still niche. And the companies that used a Swift backend overwhelmingly went with https://vapor.codes instead. It is faster, and has a nicer API.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#80

Post by sandGorgon »

bodono wrote: post #23

I'm not sure this is really going to take off, it seems that most people who are abandoning TF are moving to Jax or pytorch. My own experience with Jax is that it is much easier to use then TF, just an all round more pleasant experience. It would be interesting to try this, but at this point I'm not really willing to learn 'yet another deep learning framework' and the extreme anti-user problems that TF had make me lo…

I have just started hearing about Jax. But it seems to be a low level library that Tensorflow uses right ?

The latest release of Tensorflow probability uses JAX under the hood. So what do you mean when you say you're moving to JAX versus Tensorflow

Post reply on HN