121–130 of 432 posts
Earlier quoted context omitted.
Same questions here, Jeez. Would love to know Jeremy Howard's too.
I immediately thought of Jeremy Howard, he has mentioned this project in a couple of interviews.
No surprises there. There were maybe a total of 5 people excited about adding automatic differentiation to Swift. Too bad they didn't try improving Julia instead.
While I love using Julia for many things, I prefer having a Python monoculture for AI to be able to mix and match different algorithms. At the end the engine that compiles the mathematical expression to hardware is what matters, and I don't think that LLVM IR that uses is the best IR for the optimizations.
This is becoming increasingly outdated as significant parts in scientific machine learning require parts to be written that don't simply compile to GPUs. Think, for example, when you mix a physics model, say for RF or some such, and deep learning to model parts of the function. In python, you cannot write the RF model because python is vastly too slow, so you're forced to write the RF model in something fast, like C/C++, then integrate that to python, then integrate that to your favorite tensor network, needing more languages than you can do immediately with Julia.
Deep learning is moving rapidly out of simply being a tensor engine, and being a tool in much larger problems, where many high performance pieces need developed. Julia is light years ahead of Python for these domains, and I cannot see Python ever catching up because it suffers from performance and/or multiple language problems to solve these.
If you've never learned about scientific machine learning - go read some or watch some videos. It's fascinating and growing rapidly.
Not surprised that this project had trouble gaining traction given Swift's obsession with shoe-horning functional concepts into every crevice of the language, e.g., I can't get through a single Swift tutorial without having to get into closures.
The only functional features I can think of are closures, map/reduce/etc., and value types.
I have a lot of criticisms about Swift but this one seems weird/outdated to me.
It's a shame. I had high hopes at the beginning that S4TF - and the investment in Swift from Google - would help Swift break out of the iOS ghetto and cement it as a mainstream language. Swift's a delightful language to use. It has a lot of the nice things about Rust's type system, but is a heck of a lot easier to use at the expense of a bit of performance. For a lot of use cases, I think this is a great value propos…
I love all languages in the ML/Haskell tradition, but I think Julia would have been a better fit because it's dynamic yet efficient, and because it has a really decent probability/statistics/ML ecosystem already. Long term, I think it's the best replacement we have for Python in the ML world. Python has exceptional libraries but, as a language, it's a bit dated on several fronts. This has an impact on library design.…
I've been using Python for ML for the last 3 years and I've never felt this way. It might be that I'm not all about the hip new languages, but I don't really see the benefit of making Python more ML/Haskell-ish.
The ML use case for Python is roughly as follows: you design, train, and evaluate models. Then, if something is decent enough to use in production, you switch over your application to load and use that instead. I don't really see where Haskell or any language from the ML family can improve in that process.
Sure, the code that you used to implement your model may improve slightly, but I don't see that code improving significantly. The fruit of your labor is usually a protobuf file (encoding the TensorFlow graph) or whatever your framework uses to encode the model you built. The code actually surrounding it is very minimal for most use cases.
> In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks.
There's nothing stopping you from composing things in Python. But it's simply not a goal of 99% of ML libraries to be composable with other libraries. You're probably never gonna run TensorFlow and PyTorch in the same application (trust me, I've tried, it was a nightmare) and I don't see why you would compose a TensorFlow model with a PyTorch model without incurring tons of overhead in your application around gluing these two things.
Earlier quoted context omitted.
>I think Serverless would be another ideal use-case for Swift, where the productivity, clarity and correctness tools it offers would be a huge benefit. Oh yes, I would love to have Swift framework for Firebase on server, not only for iOS. Its atrocity to write the server logic in NodeJS after making the user App in Swift. Every time I switch from Swift to JS I deeply appreciate the beauty of Swift. On swift I do much…
It doesn’t solve the issue of JS but NestJS feels like a more robust way to make a server in JS - and the docs are great.
Earlier quoted context omitted.
What are some ways in which Python's type system should improve?
First, managing python dependencies and versions, current system is horrible. Focus on fixing that, then performance. Swift is much faster than Python. Third, Swift is a much advance and intelligent language to program in than Python. When I write Swift code, I feel like a PhD and when I write Python, I'm back in elementary school.
But for applications that require a GPU (i.e, most ML applications) cutting over to Swift from Python will likely win you nothing in performance, and wouldn't be worth it at all.
Swift now being a 100% Apple-sponsored & owned project again makes me a bit nervous. Anyone knows if chris latner is at least using swift in his new company ? I have the feeling swift never really worked in the server side, data science is now officially a failure, and all that is left is now a very niche market of 100% native mobile development. I love this language, but i'm eager to see it handled by a proper found…
Swift now being a 100% Apple-sponsored & owned project again makes me a bit nervous. Anyone knows if chris latner is at least using swift in his new company ? I have the feeling swift never really worked in the server side, data science is now officially a failure, and all that is left is now a very niche market of 100% native mobile development. I love this language, but i'm eager to see it handled by a proper found…
> I have the feeling swift never really worked in the server side IBM dropped Kitura more than a year ago [1]. There's Vapor [2] which seems popular, but I don't know how much. [1]: https://forums.swift.org/t/december-12th-2019/31735 [2]: https://github.com/vapor/vapor
Earlier quoted context omitted.
I love all languages in the ML/Haskell tradition, but I think Julia would have been a better fit because it's dynamic yet efficient, and because it has a really decent probability/statistics/ML ecosystem already. Long term, I think it's the best replacement we have for Python in the ML world. Python has exceptional libraries but, as a language, it's a bit dated on several fronts. This has an impact on library design.…
> Python has exceptional libraries but, as a language, it's a bit dated on several fronts. I've been using Python for ML for the last 3 years and I've never felt this way. It might be that I'm not all about the hip new languages, but I don't really see the benefit of making Python more ML/Haskell-ish. The ML use case for Python is roughly as follows: you design, train, and evaluate models. Then, if something is decen…
In Julia you can compose a custom distribution, with a bayesian model with an ODE with a neural network with unit number types with custom Julia written CUDA kernels and multithreading.
Edit: That are not designed specifically to work with each other
Can python even hope to do a fraction of that, still be fast and differentiate through everything?
Earlier quoted context omitted.
> I prefer having a Python monoculture for AI to be able to mix and match different algorithms Given how divergent the current crop of ML frameworks are, is this really a realistic expectation? Having played around with Julia and Flux for ML, I find I have to do just as much rewriting when translating e.g. TF -> Flux as TF -> PyTorch. You get some limited mixing and matching with Caffe2 Torch and TF JAX, but that bre…
I agree that there is poor interoperability between the different DL / auto-diff frameworks in python. But I suspect the GP is referring also to things like scikit learn, numpy, plotting tools, PIL, opencv, pandas, etc. that make up the python ecosystem. I know that alternatives exist in other languages, but I don't know of an overall ecosystem that is as mature and interoperable as in python.