Cool! One special case less in the Swift language (automatic differentiation).
Autodiff already landed into the upstream Swift implementation. Too late for that.
Crap :S
Let’s hope it atrophies over time.
Swift for TensorFlow Shuts Down
261–270 of 432 posts
Re: Swift for TensorFlow Shuts Down
#262Earlier quoted context omitted.
> If Python's performance in the ML space is not sufficient, then the community would have quickly moved on from it and built something better. Python's performance is sufficient when the bottleneck is actually the computation done in the accelerator. In my flavour of ML, we use small models, think 3 layer NN 64 neuron wide and in some cases a small CNN. During training, most of the models reported using Most of the…
> Most of the community finds python sufficient because they do not need to interleave training and simulating. That's kind of my point. Most of the community has models running on GPU's and don't care too much about CPU-bound workloads, training or otherwise. If you do care about that, you are in a relatively small niche of machine learning, statistically speaking. I am not denying it's existence, I'm just saying th…
I work with bog-standard deep learning and this does come up, albeit not in the research stage that most people are familiar with. The closer you get to deployment, the less adequate Python becomes and the more you struggle with artificial limitations like the GIL. https://news.ycombinator.com/item?id=20301619 had a good discussion on whether we've collectively "overfit" on this slow glue + fast matrix accelerator model.
Re: Swift for TensorFlow Shuts Down
#263Earlier quoted context omitted.
While I love using Julia for many things, I prefer having a Python monoculture for AI to be able to mix and match different algorithms. At the end the engine that compiles the mathematical expression to hardware is what matters, and I don't think that LLVM IR that uses is the best IR for the optimizations.
>At the end the engine that compiles the mathematical expression to hardware is what matters, This is becoming increasingly outdated as significant parts in scientific machine learning require parts to be written that don't simply compile to GPUs. Think, for example, when you mix a physics model, say for RF or some such, and deep learning to model parts of the function. In python, you cannot write the RF model becaus…
I’m seeing just the opposite as you: if it’s good enough to do protein folding for Google and video classification for Tesla, it’s good enough for me.
I’m sure that Julia is better for some specific tasks, for example phyisical simulations, that are important for some types of scientific tasks, but I don’t see any valid argument on why a simple tensor engine in Python is not smart enough to simulate the human brain, which is just a bunch of connected neurons.
Re: Swift for TensorFlow Shuts Down
#264Earlier quoted context omitted.
The Uber engineering disaster story from 2 months ago seems to suggest that Apple lacks dogfooding Swift internally: https://news.ycombinator.com/item?id=25373462
More like Uber spent too little time trying to keep their app development process manageable.
Uber doesn't have any pressure to have a reasonable tech stack because they don't have to be profitable (because they're an investor charity). On the other hand, having a fancy tech stack helps with recruiting.
Re: Swift for TensorFlow Shuts Down
#265I'm curious which of the many "evolutions" that went into Swift in the last few years were motivated heavily by its dance with TensorFlow. If there were any, can we take them back out? Now that the marriage is over?
One of the things that frustrates me with a lot of the languages I'm having to grok these days, is they often lack consistency, because it's one carrot after another to get buy in from different sub communities. And in the end it tastes like vegetable soup--edible, but not tasty (if you love vegetable soup, this reach of an analogy probably won't work for you).
Re: Swift for TensorFlow Shuts Down
#266And TensorFlow for C# is alive and kicking: https://losttech.software/gradient.html I guess this is sort of an ad. Swift was a weird choice for statically typed TensorFlow, being only popular on the platform, that does not have GPU/TPU support in TensorFlow, which is, basically, a requirement for any serious work. The fact, that they had to fork the compiler did not help either. TensorFlow for C# is much like TensorF…
> Swift was a weird choice for statically typed TensorFlow, being only popular on the platform, that does not have GPU/TPU support in TensorFlow, which is, basically, a requirement for any serious work. The fact, that they had to fork the compiler did not help either.
This is available now (not sure if it's in mainline).
https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...
Re: Swift for TensorFlow Shuts Down
#267Earlier quoted context omitted.
They have good defaults and Xcode is a good IDE with full support of these frameworks, Swift is a good language with sensible conventions so you don't really need to learn that much. When you don't know something you can start typing keywords and see what Xcode autocompletes for and read the explanation about that function from the documentation that is provided to you right there. It's not perfect, sometimes there's…
You get the same with JavaScript if you use any mainstream editor with TS support (you don’t need to be using TS yourself).
I use VSCode, it’s helpful but It’s not remotely the same. With TS it’s better when the devs included documentation.
Re: Swift for TensorFlow Shuts Down
#268Stupid question. Why is it not "TensorFlow for Swift"?
I had the same question for the naming of Windows Subsystem for Linux (WSL), which IMO should be named "Linux Subsystem for Windows"
The best explanation I can think of is that the people naming it wanted the first word to emphasize the thing that's most important to them. In Microsoft's case, that's of course Windows. Perhaps the same is true for the S4TF folks.
Re: Swift for TensorFlow Shuts Down
#269Earlier quoted context omitted.
But to some extent isn't it always going to be bolted-on and dependent on library authors to which extent it's consistent?
For what it’s worth, TypeScript had somewhat of the same problem, and yet these days it’s rare for me to find a well known library that doesn’t include type definitions!
Yeah the JS community’s adoption of typing seems to be far more enthusiastic than the Python communitys’
Re: Swift for TensorFlow Shuts Down
#270Earlier quoted context omitted.
I love Julia in general, but yeah, I hate `end`. Re: 0-indexing vs 1-indexing. If you use 0-indexing, you turn off a lot of non-engineering scientific programmers. My personal experience is that 0-indexing is better for more engineering applications, while 1-indexing is better for math. I'm a weirdo in that I don't seem to mind either one though.
I don't understand why Julia didn't do like Fortran and Ada and allow any starting index. E.g., a(-5:5, 0:3).
In case you didn’t know, Julia does in fact have arbitrary indexing!