Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift for TensorFlow Shuts Down

github.com

Post reply on HN
Print view View original

391–400 of 432 posts

Re: Swift for TensorFlow Shuts Down

#391

Post by jaegerpicker »

pjmlp wrote: post #334
jaegerpicker wrote: post #87

Earlier quoted context omitted.

Right... Because Apple has done that so many times... oh wait they have done it exactly once since re-launching after acquiring NeXT. IT was Obj-C and now it's Swift except they actively support Obj-C still, so I'm not sure what you are talking about?

Apparently you need to count better. Java was introduced alongside Objective-C, with a common bridge to call into Objective-C runtime, as Apple was unsure if the Apple developer community would be welcoming to Objective-C. They dropped it after the community was more than happy to adopt Objective-C. About the same time they introduced PyObjC support and for a while there was MacRuby, which due to internal politics wa…

Nice shifting of the goal posts. Introducing a possible technology while supporting your main one is hardly the same thing. Since Next became OS X Objective-C has been supported and it has been the main focus of development until Swift. Apple is a different company after Next, the System days have no real bearing on modern Apple. Java, PyObjC, and MacRuby were all attempts to offer more ways into the main eco-system. They were never attempts to support a new dev environment. Swift is the only time Apple has done that since Jobs returned.

Re: Swift for TensorFlow Shuts Down

#392

Post by GregarianChild »

pjmlp wrote: post #390
GregarianChild wrote: post #370

Earlier quoted context omitted.

I agree from a technical POV, but there is a social reason (network effect): Swift has Apple-backing, OCaml is not currently supported by any of the big, influential software companies. OCaml's main backer now is Jane Street who do a lot, but is too small, and Facebook's support (via ReasonML) is too half-hearted to be compelling. As a former OCaml programmer, I would not currently bank a career or startup on OCaml.…

> ...F#/C# being JIT'ed... Nope, NGEN exists since .NET 1.0, .NET Native since Windows 8, Mono aot since ages, and then there are the third party tooling like IL2CPP.

Thanks. I was not aware of this, since I've not worked on a Microsoft stack for a long time. Are those F# AOT compilers mature and support all features, including libraries? In this case F# would be a viable alternative to OCaml for Jane-Street-like tasks.

Re: Swift for TensorFlow Shuts Down

#393

Post by dragonwriter »

dagmx wrote: post #383
dragonwriter wrote: post #360

Earlier quoted context omitted.

> Is the Python type system improving or is it the type annotation system that's improving? Both. (The type annotation system is deeply tied to the type system, since the latter is what is statically verified, but there are improvements both in what can be checked—the type system—and how that is expressed/annotated.)

Do you have examples of how the type system (not annotations) have been improved?

It's important to understand that the type systems for Python are not really an inherent feature of Python, but the systems enforced by each of the static checkers (which tend to adopt each other's advancements and drive syntactic support into the core system of annotations, so there's a common baseline advancing with the individual systems). A recent advancement in one of those type systems would be pylance (actually pyright, since that's the typechecker for the pylance language server) support for higher-kinded type variables [0]. Mypy, I believe, has identified this as a thing to support, too.

An example of an improvement in annotations specifically to enable type system improvements that require supporting syntax improvement is PEP 646 support for Variadic Generics. [1] Even though it is in draft, pyright has already landed support for it for the type system improvements motivating it. [2]

[0] item #4, https://devblogs.microsoft.com/python/pylance-introduces-fiv...

[1] https://www.python.org/dev/peps/pep-0646/

[2] https://github.com/microsoft/pyright/releases/tag/1.1.107

Re: Swift for TensorFlow Shuts Down

#394

Post by eigenspace »

skohan wrote: post #344
eigenspace wrote: post #281

Earlier quoted context omitted.

are comparison operators for checking if something is less than or greater than something else. This is okay in a static language where there are special slots for a type where only a restricted subset of things can happen. You'd know at the parser level that the are referring to the type meaning or the operator meaning. In julia, types are values and values can live in types, and arbtrary operations can happen in a…

I feel like this is a solvable problem. For instance you could require parens in the case of expressions using characters.

Or you could just not use them for type parameters. That's also a solution for what it's worth. I think the curly braces look better as type params anyways imo.

Re: Swift for TensorFlow Shuts Down

#395

Post by johnmyleswhite »

xiphias2 wrote: post #263
ChrisLomont wrote: post #122

Earlier quoted context omitted.

>At the end the engine that compiles the mathematical expression to hardware is what matters, This is becoming increasingly outdated as significant parts in scientific machine learning require parts to be written that don't simply compile to GPUs. Think, for example, when you mix a physics model, say for RF or some such, and deep learning to model parts of the function. In python, you cannot write the RF model becaus…

I’m seeing just the opposite as you: if it’s good enough to do protein folding for Google and video classification for Tesla, it’s good enough for me. I’m sure that Julia is better for some specific tasks, for example phyisical simulations, that are important for some types of scientific tasks, but I don’t see any valid argument on why a simple tensor engine in Python is not smart enough to simulate the human brain,…

This comment seems a bit odd given that the connections between physical neurons are qualitatively very dissimilar from the connections in artificial neural networks, so invoking the human brain seems like a red herring in discussing AI methods.

Re: Swift for TensorFlow Shuts Down

#396

Post by dagmx »

dragonwriter wrote: post #393
dagmx wrote: post #383

Earlier quoted context omitted.

Do you have examples of how the type system (not annotations) have been improved?

It's important to understand that the type systems for Python are not really an inherent feature of Python, but the systems enforced by each of the static checkers (which tend to adopt each other's advancements and drive syntactic support into the core system of annotations, so there's a common baseline advancing with the individual systems). A recent advancement in one of those type systems would be pylance (actuall…

You're continuing to describe improvements in the annotation checking ecosystem.

I'm specifically asking about improvements in the type system. Again, two very different things that shouldn't be conflated.

Re: Swift for TensorFlow Shuts Down

#397

Post by ChrisLomont »

xiphias2 wrote: post #263
ChrisLomont wrote: post #122

Earlier quoted context omitted.

>At the end the engine that compiles the mathematical expression to hardware is what matters, This is becoming increasingly outdated as significant parts in scientific machine learning require parts to be written that don't simply compile to GPUs. Think, for example, when you mix a physics model, say for RF or some such, and deep learning to model parts of the function. In python, you cannot write the RF model becaus…

I’m seeing just the opposite as you: if it’s good enough to do protein folding for Google and video classification for Tesla, it’s good enough for me. I’m sure that Julia is better for some specific tasks, for example phyisical simulations, that are important for some types of scientific tasks, but I don’t see any valid argument on why a simple tensor engine in Python is not smart enough to simulate the human brain,…

>if it’s good enough to do protein folding for Google and video classification for Tesla, it’s good enough for me.

That's funny, since protein folding uses the methods I described to achieve it's results. Multiple stages in their work [1] uses physical modeling, not NNs, to do protein folding, most likely because the problem becomes currently intractable to solve with simple Python + TF. NNs are a step to adjust the physical model, exactly like I described above.

Simply read their paper and note all the physics models incorporated at about every step of the process to enforce physical constraints - this means vastly less parameters, less training time, less training data, faster evolution of the process, etc.

Here's their repo [2]. They did that work in Python and TF, likely because it was started years ago. As Julia becomes a much faster develop tool for this type of work I expect this will change. They also used TF 1.14 - showing the age of their development. They did not release the feature generation code which is a significant component; the released code only works on the specific dataset they provide. This is likely because this component is not simply simple python they wrote, but an amalgam of things written to make the physics parts of the chain fast enough. But they don't clearly state either way.

Also, by your argument, since it's possible to solve any NN problem with a network only 3 layers deep, why not just claim that's all one needs? Because it's also not computationally feasible.

The point is that by adding outside knowledge, such as physics models, you can have vastly smaller networks, require less training data, train and infer faster, with the end result of being able to solve a much larger class of problems efficiently.

So yes, you can do it in python, or any language, if you want to waste orders of magnitude more effort and resources to do it, effectively limiting the things you can practically do.

This is why Python incurs a unnecessary cost for such development.

By willfully ignoring learning about these methods and being ignorant about even the results you cited you will miss out on extremely useful knowledge.

[1] https://www.nature.com/articles/s41586-019-1923-7.epdf?autho...

[2] https://github.com/deepmind/deepmind-research/tree/master/al...

Re: Swift for TensorFlow Shuts Down

#398

Post by uryga »

pjmlp wrote: post #339
uryga wrote: post #253

Earlier quoted context omitted.

> I can't speak to why run-time checking wasn't built into the language as a default behavior tldr: you other get either huge overhead or need to type annotate everything like it's Java, and it'll still be pretty limited/not super useful --- there's a bunch of libraries that do this, and they're all fundamentally limited [though i guess some of these may just be failures of imagination on my part]: - hard to check ge…

> tldr: you other get either huge overhead or need to type annotate everything like it's Java, and it'll still be pretty limited/not super useful Still using Java 8 are we?

i'm aware Java has type inference now. but i've got some not-so-fond memories of learning it when it didn't :)

Re: Swift for TensorFlow Shuts Down

#399

Post by dragonwriter »

dagmx wrote: post #396
dragonwriter wrote: post #393

Earlier quoted context omitted.

It's important to understand that the type systems for Python are not really an inherent feature of Python, but the systems enforced by each of the static checkers (which tend to adopt each other's advancements and drive syntactic support into the core system of annotations, so there's a common baseline advancing with the individual systems). A recent advancement in one of those type systems would be pylance (actuall…

You're continuing to describe improvements in the annotation checking ecosystem. I'm specifically asking about improvements in the type system. Again, two very different things that shouldn't be conflated.

Oh, okay, the problem here is that we have different understanding of the relevant meanings of “type” and therefore “type system”, which is unsurprising because the term has many different uses in programming.

I am using “type” in the standard computer science sense of statically enforced invariant and “type system” as the system of statically enforced invariants, distinct from annotations which are the syntax by which types are expressed.

You seem to be referring to “types” and “type system” to refer to something else, perhaps some aspect run-time behavior / “dynamic typing”. whatever it is, there's probably also some improvement in that, too, but given that the original concept of the uphtread comment about type system improvements was about type safety, which is what the system of types that are statically verified addresses, it's not really a germane issue.

Re: Swift for TensorFlow Shuts Down

#400

Post by dagmx »

dragonwriter wrote: post #399
dagmx wrote: post #396

Earlier quoted context omitted.

You're continuing to describe improvements in the annotation checking ecosystem. I'm specifically asking about improvements in the type system. Again, two very different things that shouldn't be conflated.

Oh, okay, the problem here is that we have different understanding of the relevant meanings of “type” and therefore “type system”, which is unsurprising because the term has many different uses in programming. I am using “type” in the standard computer science sense of statically enforced invariant and “type system” as the system of statically enforced invariants, distinct from annotations which are the syntax by whi…

There's multiple aspects to what a type system can entail.

There's dynamic/static typing which is what most of the thread is about. Python is dynamically typed , which makes it really easy to pickup and make stuff in, but you get issues like runtime errors/bugs since there's no inherent type checking other than attribute access failure, or explicit isinstance calls. To the best of my knowledge, there's no advancement here in Python since it's antithetical to the language design. It is strongly typed however which is great, but not really part of the discussion since it's being assessed in the thread on the virtues of static/dynamic typing and not weak/strong typing.

Type annotation helps partially here since you can statically check the types notated post facto but this is not the type system. The type system in a language is a contract by the compiler/interpreter to enforce type safety. Annotations have no contract for correctness of description or even guarantees of being checked.

This is why I'm trying to draw the distinction between the type system vs the type annotation system. The type system itself hasn't improved much/at all in years by design. The type annotation system has however improved significantly.

Post reply on HN