Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow – A system for deep learning and differentiable computing

tensorflow.org

Post reply on HN
Print view View original

91–100 of 142 posts

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#91

Post by microtonal »

anoncept wrote: post #52
cs702 wrote: post #32

Earlier quoted context omitted.

A Rust for TensorFlow (and/or a "RustTorch") would be awesome . I hope all the work being done on improving incremental compilation[a] and developing interactive Rust REPLs like evcxr[b] makes using Rust for AI a practical reality. [a] https://doc.rust-lang.org/edition-guide/rust-2018/the-compil... [b] https://github.com/google/evcxr

As a starting point, maybe take a look at the 'tch' crate [1] and the 'rust-bert' crate [2] built on top of it? [1]: https://github.com/LaurentMazare/tch-rs -> https://crates.io/crates/tch [2]: https://github.com/guillaume-be/rust-bert -> https://crates.io/crates/rust-bert

tch-rs is really nice. I have built a Rust sequence labeler + dependency parser + lemmatizer on top of it. It supports multi-task learning, finetuning of BERT-like models, model distillation, etc.:

https://github.com/stickeritis/sticker2/

Unfortunately, when I started this guillaume-be's rust-bert crate wasn't around yet (I think), so I also ported some of the models from Huggingface transformers:

https://github.com/stickeritis/sticker-transformers/

At any rate, I can highly recommend the tch crate if you are looking to build neural networks in Rust.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#92

Post by devmunchies »

kgwgk wrote: post #83
rtorr wrote: post #15

Earlier quoted context omitted.

“Stillborn” is a pretty awful term to use for software.

Now that I think of it, does anyone know if "abort" is among the computing terms to be avoided (like master, black list or sanity check)?

Anyone who is avoiding "master, black list or sanity check" probably thinks abortion is super awesome and that the term "abort" should never be stigmatized.

Best not to worry about such silly things and keep writing code.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#93

Post by c16a »

yunohn wrote: post #30
VHRanger wrote: post #3

Who would want to use an Apple-centric language for ML, seriously? Apple hardware is outright incompatible to the kind of hardware we use daily in machine learning workstations.

I agree on the hardware part. MacOS only supports AMD GPUs which are unsupported by Tensor flow...

Tensorflow can work with ROCM which is equivalent to Nvidia's CUDA.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#94

Post by hedgehog »

The name is a bit confusing. Swift for TensorFlow combines a bunch of things:

  - adding autodiff to Swift language & compiler
  - neural net construction & training API in Swift
  - low-friction Python bindings
  - low-friction C++ interop
  - ability to run neural nets TensorFlow using the C++ interop 
  - ability to alternately run neural nets directly on XLA ("X10")
The Swift changes are supposed to get mainlined and I think at least some of the stuff related to Python and C++ already are. The idea is that Swift is nice enough to cover experiment + train + embed as library even on mobile platforms. It's a big engineering project and I hope Google keeps funding the work.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#95

Post by marmaduke »

MiroF wrote: post #60
bodono wrote: post #23

I'm not sure this is really going to take off, it seems that most people who are abandoning TF are moving to Jax or pytorch. My own experience with Jax is that it is much easier to use then TF, just an all round more pleasant experience. It would be interesting to try this, but at this point I'm not really willing to learn 'yet another deep learning framework' and the extreme anti-user problems that TF had make me lo…

All I want is a way to statically type check tensor axes. Why can't I get a way to statically type check tensors?

Futhark has size types,

https://futhark-lang.org/blog/2020-03-15-futhark-0.15.1-rele...

and it seems to be ok for DL

https://elsman.com/pdf/fhpnc19.pdf

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#96

Post by mark_l_watson »

I question the long term viability of Swift for TensorFlow. I hope that I don’t sound like I am whining but I invested a fair amount of time with Swift from last fall through this February because I wanted: a better faster language to work in for DL than Python; I was also interested in trying some iOS, iPadOS, and macOS development; Swift looked interesting also for Linux server side work.

For Apple development I have found Flutter+Dart to be more pleasant. For DL I decided to stick with taking advantage of my 5+ years experience with TF in Python.

Off topic, but Julia and Flux are really worth checking out for DL.

Some advice: if you want to experiment with Swift for TensorFlow use Google’s colab and make your life easier. I wish I could get back the install times on my Linux GPU laptop and on macOS. If you pay for colab like I do, you get good GPU/TPU resources, and it is simply an easy and fun way to go.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#97

Post by marcinzm »

yyyk wrote: post #90
marcinzm wrote: post #20

Earlier quoted context omitted.

They presumably wanted a semi-popular statically typed language as the gains of Julia over Python aren't enough to be worth it (and Julia isn't popular enough).

Semi-popular where? The typical iOS programmer isn't going to touch ML. In the relevant demography, Julia had (still has) more users. Swift4TF was a poor choice, and now that Lattner is gone I doubt it has a future.

Julia is out because it's not statically typed and not enough of a difference from Python. If it was more popular then it may have had a chance on that alone but it's not. In terms of semi-popular statically typed modern languages that have good/performant C/C++ interop you have: Go, Swift and Rust. Can't think of any others to be honest. I'm guessing Go's lack of generics excluded it and Rust's complexity excluded it.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#98

Post by falseprofit »

devmunchies wrote: post #92
kgwgk wrote: post #83

Earlier quoted context omitted.

Now that I think of it, does anyone know if "abort" is among the computing terms to be avoided (like master, black list or sanity check)?

Anyone who is avoiding "master, black list or sanity check" probably thinks abortion is super awesome and that the term "abort" should never be stigmatized. Best not to worry about such silly things and keep writing code.

I don't know of anyone who "thinks abortion is super awesome"...

In any case, the primary meaning of 'abort' is more general, so it shouldn't be compared to the metaphorical use of 'stillborn'.

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#99

Post by BadInformatics »

MiroF wrote: post #60
bodono wrote: post #23

I'm not sure this is really going to take off, it seems that most people who are abandoning TF are moving to Jax or pytorch. My own experience with Jax is that it is much easier to use then TF, just an all round more pleasant experience. It would be interesting to try this, but at this point I'm not really willing to learn 'yet another deep learning framework' and the extreme anti-user problems that TF had make me lo…

All I want is a way to statically type check tensor axes. Why can't I get a way to statically type check tensors?

There are a few efforts working in this space. As you can imagine, all of them are experimental:

- Dex: https://github.com/google-research/dex-lang/ - Hasktorch: https://github.com/hasktorch/hasktorch - This initiative from the Python Typing-sig: https://docs.google.com/document/d/1oaG0V2ZE5BRDjd9N-Tr1N0IK...

Re: Swift for TensorFlow – A system for deep learning and differentiable computing

#100

Post by yyyk »

marcinzm wrote: post #97
yyyk wrote: post #90

Earlier quoted context omitted.

Semi-popular where? The typical iOS programmer isn't going to touch ML. In the relevant demography, Julia had (still has) more users. Swift4TF was a poor choice, and now that Lattner is gone I doubt it has a future.

Julia is out because it's not statically typed and not enough of a difference from Python. If it was more popular then it may have had a chance on that alone but it's not. In terms of semi-popular statically typed modern languages that have good/performant C/C++ interop you have: Go, Swift and Rust. Can't think of any others to be honest. I'm guessing Go's lack of generics excluded it and Rust's complexity excluded i…

TF's own analysis[0] was hesitant on static vs dynamic, finding downsides for either choice and suggested choosing a "middle ground" (which IMHO was doable starting from Julia's optional types).

So static typing might be your requirement, but it wasn't TF's requirement. When examining Julia later on, they don't mention dynamic types as a downside at all - and every argument save for "we're familiar with Swift's internal working" was contradicted within the same document...

Now, being familiar with Swift is a very understandable reason for their choice, but it's IMHO not the best choice for others which are not as familiar with Swift or ML. Most ML users are very familiar with Python and I think they will find Julia to be a welcome improvement on Python's pain points without Swift's baggage.

[0] https://github.com/tensorflow/swift/blob/master/docs/WhySwif...

Post reply on HN