Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow Shuts Down

github.com

Post reply on HN
Print view View original

371–380 of 432 posts

Re: Swift for TensorFlow Shuts Down

#371

Post by improbable22 »

ZephyrBlu wrote: post #367
improbable22 wrote: post #365

Earlier quoted context omitted.

Maybe less readable than I remembered, sorry! But these `CartesianIndex` things are how you index multi-dimensional arrays. If by indexing you only mean adding offsets to pointers with your bare hands, this is possible but discouraged. Because you'll produce a buggier and less general version of exactly what already exists. In this case, reversing an N-dimensional array, along any M of its axes, with arbitrary offset…

What I mean is something like this: # one dimensional a = [1, 2, 3] # prints the number 1 @ index 0 print(a[0]) # multi-dimensional b = [[1, 2, 3], [4, 5, 6]] # prints the number 4 @ index (1, 0) print(b[1][0]) In Julia these indexes would start from 1. It doesn't seem like `CartesianIndex` solves this, since looking at some examples you still have to specify array indexes (Which start from 1): https://docs.julialang…

OK, I thought you were advocating zero based on complicated index calculations for some algorithm, with mod & div, which is what I was trying to say could be abstracted away. The literal syntax `a=[1,2,3]` makes a one-dimensional `Array{Int,1}` whose first element is indeed `a[1]`. (`b` is an array of arrays, while `c = [1 2 3; 4 5 6]` is an honest 2-dimensional array.)

Re: Swift for TensorFlow Shuts Down

#372

Post by dd82 »

The_rationalist wrote: post #291
cwyers wrote: post #231

Earlier quoted context omitted.

More like... most of the scientific Python stack has settled on a third party library, NumPy, to provide support for numerical computing. That includes providing data types like arrays. Tensorflow doesn't use NumPy's data types, so if you want to use Tensorflow Probability, you need to convert everything you've done in NumPy (or things that are built on top of NumPy, like Pandas) to TensorFlow, and use TensorFlow's f…

https://www.tensorflow.org/guide/tf_numpy

yes, and if you go to that link, you can see its experimental, for TF 2.4 _and_ the notebook on github was created Aug 3, 2020. So that is relatively new and most likely has a ways to go before further adoption. Its good that option exists, but doesn't seem very backwards compatible. That doesn't negate the points made in the parent.

Re: Swift for TensorFlow Shuts Down

#373

Post by kortex »

codethief wrote: post #350
simias wrote: post #70

Earlier quoted context omitted.

I agree with you, although I must say that as an occasional Python user the way type hints are implemented absolutely baffles me. In particular the fact that they are completely ignored by the standard implementation. They're glorified comments. Given how opinionated the Python maintainers can be, it baffles me that they accepted to get these optional, noisy, half backed type hints into the core language. In my exper…

My view is completely opposite to yours. I find type hints extremely useful and every new line of Python code I write uses them. I finally no longer have to do type checking / duck typing in my head but can simply write the types down. Moreover, an IDE like PyCharm will then do the type checking for me, notifying me of potential errors. (Even if I decide not to use Mypy.) Third, I finally only need to look at the sig…

Bingo. I think a big explanation of the wide variance on opinion about type hints is due to reliance on tooling, or lack thereof.

Writing type hinted python in Pycharm or Ipython feels like racing in a sports car. Without it, feels like stop and go traffic with the occasional car accident.

I can see how if you are using a WYSIWYG editor, type hints feel like more text for little gain. But even then, it really helps document intent. In fact switching to IDEs is partly what made me go from someone who hated static typing to loving it.

Re: Swift for TensorFlow Shuts Down

#374

Post by kortex »

simias wrote: post #70
superbcarrot wrote: post #57

Earlier quoted context omitted.

It fundamentally can't be improved in a significant enough way and still be Python. The more realistic options are to be okay with Python's type system (I personally am) or to look at different languages if you really want something more robust.

I agree with you, although I must say that as an occasional Python user the way type hints are implemented absolutely baffles me. In particular the fact that they are completely ignored by the standard implementation. They're glorified comments. Given how opinionated the Python maintainers can be, it baffles me that they accepted to get these optional, noisy, half backed type hints into the core language. In my exper…

> They're glorified comments

This was always kinda false (type hints can be composed unlike eg docstring param @type). But with the introduction of dataclass, this is empirically false. You can readily hook into the type hints, reflect on them, use it to marshal data, etc. Using __post_init__, you can leverage any kind of runtime checks you want to enforce invariants.

Also type hints greatly help transpilation.

You don't need 100% to reap the benefits. In fact I will annotate small bits of un-typed code and it makes it way easier to reason about.

Re: Swift for TensorFlow Shuts Down

#375

Post by The_rationalist »

andrekandre wrote: post #343
The_rationalist wrote: post #298

Earlier quoted context omitted.

Swift has no traced garbage collector which means you cannot represent circular data structure/references. Also swift support for interfaces (protocol) is sub-par with today standards

> Also swift support for interfaces (protocol) is sub-par with today standards interesting, what could be added/changed to make it better?

No support for default implementation methods. No support for generics in protocol.

There might be workarounds but for a newcomer, this is a red flag of reinventing a basic and universal construct (interfaces) in a sub-par one, this show a dangerous mentality from the swift vm devs, what else did they wrongfully reinvent in a sub-par way?

Re: Swift for TensorFlow Shuts Down

#376

Post by gsathya_hn »

socialdemocrat wrote: post #240

Back when I was doing a lot of Swift programming, this seemed like such a great idea. Since then I moved on to Julia. And when I looked at the ML code for Swift and compared it to Julia, I was thinking "How on earth could anyone think this was ever a good idea?" Doing machine learning stuff in Julia is simply much more user friendly than doing the same in Swift. Swift is nice for iOS development, but I think in data…

> Julia uses a method JIT, which are quick to implement, but which makes latency gets bad. Meaning if you load a whole new package, running the first function can cause some delay. With a JavaScript style tracer JIT added into the mix one could have probably managed to have both high performance and low latency.

This is actually the other way around -- Method JITs are harder to implement and all the major JavaScript JITs are method JITs, not tracing JITs. There are many reasons that contribute to the latency but I don't see what makes method JITs inherently slower than tracing JITs.

Re: Swift for TensorFlow Shuts Down

#377

Post by mark_l_watson »

socialdemocrat wrote: post #223
swagonomixxx wrote: post #181

Earlier quoted context omitted.

> It's definitely not possible. These have to be rewritten with the specific autodiff /ML framework in mind. I don't understand why it's not possible. You're asking if it's possible in the language. I don't see anything stopping you from, as you say, writing your own framework, or simply doing it in TF. Perhaps my ignorance of Julia is showing :)

Admittedly it is a bit hard to see why Julia is so different in this area if you have not spent some with it. It took me a bit time as well to see. But there really is a profound difference. The thing is that Python is really just a language to rearrange networks of nodes written in C++. A node written for one ML framework will be incompatible with a node written in another language. If you want them to work together…

That was a great explanation that makes me appreciate Julia more, thanks.

I don’t much use Julia, but it is a beautiful language. I started a tiny side project for using Julia for non-numerical things like querying SPARQL endpoints, text processing, etc. Julia could be a universal, use for almost everything language.

Re: Swift for TensorFlow Shuts Down

#378

Post by johnnycerberus »

sanderjd wrote: post #368
pjmlp wrote: post #335

Earlier quoted context omitted.

Scala already has a foothold on the same research domain due to large data frameworks like Hadoop and Spark, with NVidia caring for polyglot support on CUDA, including JVM based languages.

That is true, and certainly relevant, but I believe it is also true that Scala does not meet the criteria for what that were trying to do, because it is not easy (perhaps not possible) to do the kind of static analysis necessary to achieve their goals. For a generic "what language with a good type system should we use with ML?" project, I would totally agree with you that Scala is an obvious choice. But this project…

I really don't see how Scala is not fit for this. Swift for Tensorflow was mainly revolving around the Differentiable protocol.

There is a paper published in 2019 from Purdue University where they show how AD can be combined with multi-stage programming in Scala. [1]

DeepLearning.scala that does compile-time reverse AD. [2]

Compute.scala for operator fusion at runtime (JITifying) [3]

Here [4] is a Pre-SIP to add reifiable control flow to Scala compiler so AD and operator fusion could happen at compile-time.

Moreover, Kotlin also has a library that does type-safe AD. [5]

nVidia and Oracle want better interopabiltiy between CUDA and GraalVM through grCuda. [6]

[1] Demystifying Differentiable Programming: Shift/Reset the Penultimate Backpropagator - https://arxiv.org/pdf/1803.10228.pdf

[2] https://github.com/ThoughtWorksInc/DeepLearning.scala

[3] https://github.com/ThoughtWorksInc/Compute.scala

[4] https://contributors.scala-lang.org/t/pre-sip-for-with-contr...

[5] https://github.com/breandan/kotlingrad

[6] https://developer.nvidia.com/blog/grcuda-a-polyglot-language...

Re: Swift for TensorFlow Shuts Down

#379

Post by cbolton »

socialdemocrat wrote: post #240

Back when I was doing a lot of Swift programming, this seemed like such a great idea. Since then I moved on to Julia. And when I looked at the ML code for Swift and compared it to Julia, I was thinking "How on earth could anyone think this was ever a good idea?" Doing machine learning stuff in Julia is simply much more user friendly than doing the same in Swift. Swift is nice for iOS development, but I think in data…

I like Julia a lot, and I think it's the best option for ML research, but isn't there a big advantage to Swift or similar languages when it comes to production deployments on mobiles? The distribution of compiled binaries in Julia seems a bit kludgy and less reliable than with a statically compiled language like Swift.

Re: Swift for TensorFlow Shuts Down

#380

Post by cs702 »

dnautics wrote: post #310
cs702 wrote: post #211

Earlier quoted context omitted.

Sure, but isn't it nice that you don't even have to read the paper to play with the full model, to see if the claims hold up, using the same Python tools you and everyone else uses?

To be honest that is not really true for many ML implementations from papers. Often, I will pull a repo and it will be half-baked, or, only works on python 2.4, or have a broken dependency on something or another (zmq was common issue for a hot sec), or not publish weights, or be in tf when I want torch, or be in torch when I want tf, etc.

Yes, I agree: a lot of code is indeed half-baked! But good papers tend to have better-than-average code, and ~100% of the time, the code is in Python, not some other language.
Post reply on HN