Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow Shuts Down

github.com

Post reply on HN
Print view View original

181–190 of 432 posts

Re: Swift for TensorFlow Shuts Down

#181

Post by swagonomixxx »

amkkma wrote: post #161
swagonomixxx wrote: post #145

Earlier quoted context omitted.

I'm fairly certain everything you said is possible except for custom CUDA kernels in pure Python. You'd have to write the kernel in C++ and use it in your TensorFlow/PyTorch code. [0][1] [0]: https://www.tensorflow.org/guide/create_op [1]: https://pytorch.org/tutorials/advanced/cpp_extension.html

It's definitely not possible. These have to be rewritten with the specific autodiff /ML framework in mind. Even then, you're not going to have fast custom types to be used on the GPU without dropping into C++

> It's definitely not possible. These have to be rewritten with the specific autodiff /ML framework in mind.

I don't understand why it's not possible. You're asking if it's possible in the language. I don't see anything stopping you from, as you say, writing your own framework, or simply doing it in TF. Perhaps my ignorance of Julia is showing :)

Re: Swift for TensorFlow Shuts Down

#183

Post by swagonomixxx »

amkkma wrote: post #166
swagonomixxx wrote: post #141

Earlier quoted context omitted.

Sure, that's fine. But I'm assuming GGP is talking about TensorFlow, since that's what this post is about. If you don't need TensorFlow, then this whole conversation is kinda moot, and you can use whatever you like.

Nobody needs tensorflow. That's just a kludge around the fact that python is terrible for ML.

So nobody needs TensorFlow, yet it and PyTorch are pretty much the only frameworks widely adopted for ML, and they're in Python. A quick Google search will tell you that, I don't really feel like rebutting to a baseless statement.

Re: Swift for TensorFlow Shuts Down

#184

Post by _qulr »

rjmccall wrote: post #167
_qulr wrote: post #131

Earlier quoted context omitted.

Fair criticism. I'm not sure why the author didn't try to detect the use of objc_msg functions, for example. So the ObjC binaries may be overcounted a bit. Still, the test for Swift binaries seems accurate, and if you look at iOS 13.1 vs 14.0, for example, according to the chart there was an increase of 157 Swift binaries and 446 Objective-C binaries. If we assume there are 157 "false positives" in the ObjC binaries,…

Swift frameworks that use ObjC system libraries will still use objc_msgSend.

D'oh, right. A good test might be difficult. In the absence of that, I guess the safe bet is just to count every Swift binary as a false positive for ObjC, though that's not quite fair, since you can mix the two in the same code base.

Re: Swift for TensorFlow Shuts Down

#185

Post by belval »

onelovetwo wrote: post #153
cs702 wrote: post #119

S4TF lost out, not to Python, but to Python's AI/ML ecosystem -- people, projects, libraries, frameworks. Despite its many shortcomings, Python has become the lingua franca of AI and ML, to the point that whenever I come across a newly published AI or ML paper that I find interesting, I expect to be able to find code implementing it in Python. For example, yesterday I saw a post on HN about approximating self-attenti…

Its becoming like the same grasp js has on the web...which is sad to see.

This is a false equivalence of the first order. What makes JS impossible to dethrone is that whatever replaces it would need to be supported by a critical mass of web browsers, which just won't happen.

AI/ML on the other hand has no such restriction. Your new startup could build all their ML models in Java with DeepLearning4J and it would probably work fine. Python is used because it's easy to work with for experimenting, which is a big part of research and for production your can extract your weight and graph structure to run on TVM/TensorRT/ONNX. There is no equivalent vendor lock-in.

Re: Swift for TensorFlow Shuts Down

#186

Post by simonbarker87 »

moocowtruck wrote: post #125
simonbarker87 wrote: post #76

Earlier quoted context omitted.

It doesn’t solve the issue of JS but NestJS feels like a more robust way to make a server in JS - and the docs are great.

nestjs is an atrocity that should not have been wrought upon the js/ts community..

Sorry you feel that way, it feels like angular for the backend to me which is nice, it follows logical patterns, loading modules allows us to configure micro services from a single mono repo, decorators are nice, encourages use of RxJS which is great and it still allows access to the underlying express app if needs be.

Might not be to everyone’s tastes but it feels like a solid tool to me.

Re: Swift for TensorFlow Shuts Down

#187

Post by xenihn »

BadInformatics wrote: post #5

Looks like @throw6606 was right: https://news.ycombinator.com/item?id=24533937 . Does anyone know the status of https://ai.facebook.com/blog/paving-the-way-for-software-20- ...?

Reading that thread, I'm always so confused by HN posters who ask for evidence that would be impossible to provide without self-incriminating. Are they asking disingenuously, or are they just clueless?

Re: Swift for TensorFlow Shuts Down

#188

Post by swagonomixxx »

PartiallyTyped wrote: post #148
swagonomixxx wrote: post #117

Earlier quoted context omitted.

I'm not sure I understand what your getting at. If Python's performance in the ML space is not sufficient, then the community would have quickly moved on from it and built something better. And that something better is certainly not Julia and it's definitely not Swift. > Yes, I know the heavy lifting is done by C/C++/Rust/Cuda/Blas/Numba and so on, but, when you run simulations for millions of steps, you end up with…

> If Python's performance in the ML space is not sufficient, then the community would have quickly moved on from it and built something better. Python's performance is sufficient when the bottleneck is actually the computation done in the accelerator. In my flavour of ML, we use small models, think 3 layer NN 64 neuron wide and in some cases a small CNN. During training, most of the models reported using Most of the…

> Most of the community finds python sufficient because they do not need to interleave training and simulating.

That's kind of my point. Most of the community has models running on GPU's and don't care too much about CPU-bound workloads, training or otherwise.

If you do care about that, you are in a relatively small niche of machine learning, statistically speaking. I am not denying it's existence, I'm just saying that your stack will have to be different if you want to extract the maximum level of performance out of your CPU.

> IIRC, a maintainer of a popular project/framework that I contribute saw improvements upwards of 2x when using C++ over python.

That's not surprising at all. Like I mentioned in the parent, if you profiled your code and you found that Python function calls are the main bottleneck, and you believe it to be worth investing time in getting rid of them, you can use the C++ API's of Caffe/TF/PyTorch/Whatever.

I personally don't work in simulations so I haven't ran into your problem. In the deep learning world, the CPU is unusable for any task (training, inference, evaluation, etc.), so I've never been concerned with things like function call overhead.

Re: Swift for TensorFlow Shuts Down

#189

Post by swagonomixxx »

skohan wrote: post #151
swagonomixxx wrote: post #136

Earlier quoted context omitted.

> Having a half way decent type system solves so many problems What problems does it solve for you? When building ML applications I mean. Because that's what S4TF was supposed to be. ML researchers produce models that are used by ML applications for a particular purpose. The code to train the model _could_ be statically typed, sure, but I really don't see what the improvement would be. It would be more verbose, less…

My guess is that's because ML researchers typically have not spent much time with statically typed languages. I was also once convinced that static typing was not so valuable - when I was working a lot with js, python and ruby, but the more time I spend with static typing the more I like it. There is an "activation energy" to overcome with static typing: when you first start it feels like a tedious burden to have to…

I am totally with you on statically typed vs. dynamically typed languages _in general_. I'm not denying that there are major advantages to using one for most kinds of programs. But I just don't see a benefit for ML.

You can easily find out if you like it or not, you can use the C++ API for TF/PyTorch and you'll have a statically typed program running your model for you. Sure, it's not Haskell, but C++ is statically typed, and although it's not the most popular or loved language on HN, you can use it to get stuff done.

Re: Swift for TensorFlow Shuts Down

#190

Post by spamizbad »

nextos wrote: post #60
skohan wrote: post #26

It's a shame. I had high hopes at the beginning that S4TF - and the investment in Swift from Google - would help Swift break out of the iOS ghetto and cement it as a mainstream language. Swift's a delightful language to use. It has a lot of the nice things about Rust's type system, but is a heck of a lot easier to use at the expense of a bit of performance. For a lot of use cases, I think this is a great value propos…

I love all languages in the ML/Haskell tradition, but I think Julia would have been a better fit because it's dynamic yet efficient, and because it has a really decent probability/statistics/ML ecosystem already. Long term, I think it's the best replacement we have for Python in the ML world. Python has exceptional libraries but, as a language, it's a bit dated on several fronts. This has an impact on library design.…

Julia gets compared a lot to Python but truthfully it strikes me as more like a replacement for FORTRAN, MATLAB, and R. The ergonomics of the language seem good but documentation is poor and as a community it's narrowly scoped into the ML/numerical processing world.

If all you do is ML/numerical processing, against data that's already been cleaned up, I bet it's really great tho.

Post reply on HN