Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow Shuts Down

github.com

Post reply on HN
Print view View original

221–230 of 432 posts

Re: Swift for TensorFlow Shuts Down

#221

Post by renerthr »

Karrot_Kream wrote: post #101
nextos wrote: post #60

Earlier quoted context omitted.

I love all languages in the ML/Haskell tradition, but I think Julia would have been a better fit because it's dynamic yet efficient, and because it has a really decent probability/statistics/ML ecosystem already. Long term, I think it's the best replacement we have for Python in the ML world. Python has exceptional libraries but, as a language, it's a bit dated on several fronts. This has an impact on library design.…

> In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks. I was taken aback when looking at Turing for Bayesian modelling that the distributions were just the standard distributions found in the Distributions package! In Python, every Bayesian fra…

> but it all composes in Julia.

You mean that, in Julia, we the users have to "compose" our own implementations of models (e.g. log probabilities), as opposed to using the already-made ones in Python?

Re: Swift for TensorFlow Shuts Down

#222

Post by anon_tor_12345 »

swagonomixxx wrote: post #145
amkkma wrote: post #129

Earlier quoted context omitted.

>there's nothing stopping you from composing things in Python. In Julia you can compose a custom distribution, with a bayesian model with an ODE with a neural network with unit number types with custom Julia written CUDA kernels and multithreading. Edit: That are not designed specifically to work with each other Can python even hope to do a fraction of that, still be fast and differentiate through everything?

I'm fairly certain everything you said is possible except for custom CUDA kernels in pure Python. You'd have to write the kernel in C++ and use it in your TensorFlow/PyTorch code. [0][1] [0]: https://www.tensorflow.org/guide/create_op [1]: https://pytorch.org/tutorials/advanced/cpp_extension.html

It is possible

https://numba.pydata.org/numba-doc/latest/cuda-reference/ker...

which basically calls NVPTX

https://github.com/numba/numba/blob/ec6fa07c12c3703a52c2b4ac...

which is exactly what CUDA does anyway; CUDA code is actually a frontend for an ISA

https://llvm.org/docs/NVPTXUsage.html

Re: Swift for TensorFlow Shuts Down

#223

Post by socialdemocrat »

swagonomixxx wrote: post #181
amkkma wrote: post #161

Earlier quoted context omitted.

It's definitely not possible. These have to be rewritten with the specific autodiff /ML framework in mind. Even then, you're not going to have fast custom types to be used on the GPU without dropping into C++

> It's definitely not possible. These have to be rewritten with the specific autodiff /ML framework in mind. I don't understand why it's not possible. You're asking if it's possible in the language. I don't see anything stopping you from, as you say, writing your own framework, or simply doing it in TF. Perhaps my ignorance of Julia is showing :)

Admittedly it is a bit hard to see why Julia is so different in this area if you have not spent some with it. It took me a bit time as well to see.

But there really is a profound difference. The thing is that Python is really just a language to rearrange networks of nodes written in C++. A node written for one ML framework will be incompatible with a node written in another language. If you want them to work together you need to spend a lot of effort gluing them together, and most likely the performance benefits are lost.

This is very different in Julia because you are using pure Julia code. What is specialized C++ nodes in Python, are for the most part just plain functions in Julia. There is nothing special about them. That is why they can be used anywhere.

In Python frameworks each of these C++ nodes need to contain information about how auto-differentiation is done. That is why they are tailor made for one specific ML library.

In Julia the whole automatic differentiation stuff is perform code transformations on regular Julia code.

Are you familiar with LISP, and the idea of code as data? That is what we are talking about here. A regular syntax graph of regular Julia code gets manipulated to do autodiff. You are not organizing C++ nodes in a network.

Re: Swift for TensorFlow Shuts Down

#224

Post by jl2718 »

amkkma wrote: post #161
swagonomixxx wrote: post #145

Earlier quoted context omitted.

I'm fairly certain everything you said is possible except for custom CUDA kernels in pure Python. You'd have to write the kernel in C++ and use it in your TensorFlow/PyTorch code. [0][1] [0]: https://www.tensorflow.org/guide/create_op [1]: https://pytorch.org/tutorials/advanced/cpp_extension.html

It's definitely not possible. These have to be rewritten with the specific autodiff /ML framework in mind. Even then, you're not going to have fast custom types to be used on the GPU without dropping into C++

Maybe Numba with Enzyme will catch up to Julia’s features before Julia catches up to Python’s users.

I think the reason why people use python is because it’s the only universally-accepted language for coding interviews.

Re: Swift for TensorFlow Shuts Down

#225

Post by socialdemocrat »

selectodude wrote: post #146
amkkma wrote: post #129

Earlier quoted context omitted.

>there's nothing stopping you from composing things in Python. In Julia you can compose a custom distribution, with a bayesian model with an ODE with a neural network with unit number types with custom Julia written CUDA kernels and multithreading. Edit: That are not designed specifically to work with each other Can python even hope to do a fraction of that, still be fast and differentiate through everything?

The outcome is more important than the code. While that sounds very elegant, what does it actually add?

The Julia approach gives quicker development. Some of these ML libraries such as Flux are so small that almost anyone can read the code and learn how it works and make modifications. It is not for anyone to jump into TensorFlow.

The irony here is that tiny Julia libraries give the same power as much larger Python libraries which require highly specialized and trained developers to evolve and maintain.

With Julia it is much easier to grow the eco system because you can make lots of relatively small packages which can then be combined in almost any way.

The challenge for the Julia community today is really to make people new to the environment aware of this.

I have encountered people who thought Flux couldn't do anything because it was so small. He was so used to the huge monolithic libraries in Python, that it did not occur to him that adding something like an new activation function is literally one line of code. He is used to that requiring adding various C++ nodes. Wrap those up and God knows what more steps you need when you extend something like TensorFlow.

I don't think most people realize how powerful Julia is. They are not used to being able to combine libraries the way you can do in Julia. A big part of the effort for the Julia community will simply be to write more tutorial and introduction which better introduce beginners to these kinds of capabilities. Once you know Julia it is pretty obvious.

But grab people's attention and make them realize Julia could be a solution to their problem, I think one needs a lot of sort of shallow and quick intros to these kinds of things.

Re: Swift for TensorFlow Shuts Down

#226

Post by liuliu »

mark_l_watson wrote: post #215

I am disappointed, but so it goes. I started writing a Swift AI book a while back and I had planned on abut 20% of the book would be on S4TF. I still have lots of other material, but obviously I am not going to include S4TF material. BTW, the Haskell bindings for TensorFlow have been actively maintained for years. The latest update was 3 days ago.

It is much easier to maintain a language binding than a full toolchain. There are a lot of activities in Swift mainline, and I can only imagine how much work it must be to maintain the S4TF fork.

Re: Swift for TensorFlow Shuts Down

#227

Post by tmp538394722 »

> Added language-integrated differentiable programming into the Swift language. This work continues in the official Swift compiler.

added lots of novel paradigms and rarely trodden code paths to the compiler and then peaced out.

I hope the inherent complexity added doesn’t impede future Swift development much, or is expeditiously deleted.

Re: Swift for TensorFlow Shuts Down

#228

Post by saagarjha »

stephencanon wrote: post #163
ChrisMarshallNY wrote: post #152

Earlier quoted context omitted.

Probably right, but ObjC inherits C (and ObjC++ inherits C++), so you get all that low-level goodness, as well as the SmallTalk stuff.

You can more or less write C in Swift as well, though. Some pointer operations get more verbose, but the standard library also makes some stuff easier. The only thing that's really missing is a nice fixed-size array syntax (they get imported from C as tuples, which is not ideal).

Const and variadic generics would be nice, yes :)

Re: Swift for TensorFlow Shuts Down

#229

Post by saagarjha »

aplummer wrote: post #192
ljm wrote: post #147

Earlier quoted context omitted.

I do. Scala is insanely complicated once you inherit a codebase full of implicits and custom operators.

If misuse of a language feature is a criticism of the language, I have bad news about javascript.

Scala seems to encourage misuse :(

Re: Swift for TensorFlow Shuts Down

#230

Post by saagarjha »

coolspot wrote: post #46
saagarjha wrote: post #35

Earlier quoted context omitted.

LLVM? WebKit?

WebKit started as KDE’s KHTML/KJS and LLVM started as a research project at Univeristy of Illinois. Apple took the code and extended it, they had little choice for the source code license. https://en.wikipedia.org/wiki/Webkit https://en.wikipedia.org/wiki/LLVM

They could have just not taken the code, though…
Post reply on HN