Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gluon – Deep Learning API from AWS and Microsoft

aws.amazon.com

Post reply on HN
Print view View original

31–40 of 85 posts

Re: Gluon – Deep Learning API from AWS and Microsoft

#31

Post by grtrans »

staticelf wrote: post #21
pm90 wrote: post #18

Earlier quoted context omitted.

Mostly because Python is a great "glue" language. It isn't performant enough to implement the actual low-level computation, but is better at running other applications, getting data from them, feeding them into other applications (a.k.a pipelines).

Sure but if I have invested a lot in MS technologies I am hesitant to learn and implement things in a completely new language if I can find something that is more fitting to the stack I already use.

It’s not clear eg what use you would get from sharing a language between your model training and request processing.

Re: Gluon – Deep Learning API from AWS and Microsoft

#32

Post by losteric »

staticelf wrote: post #21
pm90 wrote: post #18

Earlier quoted context omitted.

Mostly because Python is a great "glue" language. It isn't performant enough to implement the actual low-level computation, but is better at running other applications, getting data from them, feeding them into other applications (a.k.a pipelines).

Sure but if I have invested a lot in MS technologies I am hesitant to learn and implement things in a completely new language if I can find something that is more fitting to the stack I already use.

If you already have a good stack that you're very effective, the switching cost may not be worth it.

However, if you're more algorithmically focused, python is a great DSL

Re: Gluon – Deep Learning API from AWS and Microsoft

#33

Post by HelloNurse »

staticelf wrote: post #21
pm90 wrote: post #18

Earlier quoted context omitted.

Mostly because Python is a great "glue" language. It isn't performant enough to implement the actual low-level computation, but is better at running other applications, getting data from them, feeding them into other applications (a.k.a pipelines).

Sure but if I have invested a lot in MS technologies I am hesitant to learn and implement things in a completely new language if I can find something that is more fitting to the stack I already use.

What relevant stack of Microsoft technology are you hoping to leverage for high performance numerical computation? Surely nothing involving .NET. Python for Windows and Python extensions can be compiled with Visual Studio; I don't see other ways to be more Microsoft-friendly.

Re: Gluon – Deep Learning API from AWS and Microsoft

#34

Post by alexcnwy »

staticelf wrote: post #3

Can someone please shed light to why so many ML tools and frameworks are being implemented in Python? What makes Python so special for doing ML? Personally, I would love for MS to release or support a .NET based ML toolkit. There is open source stuff like http://accord-framework.net but I would assume that it isn't as big nor complete as a framework being supported by a major corporation.

Most DL researchers are more into math/stats/theory than programming and Python is faaar easier to pick up and grok than java/.net/etc.

Re: Gluon – Deep Learning API from AWS and Microsoft

#38

Post by kuschku »

grtrans wrote: post #30
bertomartin wrote: post #27

Earlier quoted context omitted.

Can you be specific on what makes this a headache? Your "refactoring" tells me that the code probably wasn't well structured in the first place and if so this would make refactoring difficult for any language, particularly dynamically typed ones.

> Your "refactoring" tells me that the code probably wasn't well structured in the first place Well considering this is a realistic scenario for fallible humans, it’s still decent advice to keep your exploratory projects in python small to avoid ridiculous tech debt. It’s not quite as bad as with ruby, but it’s close.

I've got exactly that experience.

Many languages make it problematic to keep code actually bug-free and maintainable, and Python and especially Ruby are problematic for that, while Java and Kotlin, but even C++ (with a strict style guide) are a lot nicer to work with at scale.

If you want to keep consistent APIs between modules, strict types and checked exceptions are very helpful, while with python one typo can lead to accesses being lost — which is why so many use slots nowadays, and TypedPython, and annotations. But if I do that, I might as well use Java or Kotlin, and get a better IDE.

Compared to unit tests, strict and static types are faster, compared to no testing, static types are safer.

Re: Gluon – Deep Learning API from AWS and Microsoft

#39

Post by chirau »

kuschku wrote: post #25
aabajian wrote: post #24

Earlier quoted context omitted.

As a long-time Java developer, Python was a beauty. It brings back the joy of programming and makes data manipulation a breeze. C# is better than Java, but it's still not as elegant/simple/clean as Python for data science.

Python is only fun for tiny projects. Once you reach 120k LOC in a project, refactoring in Python is an insanity even with PyCharm, and debugging becomes impossible, too. Have you tried Kotlin?

Data Engineer here... How do you get to 120K LOC without splitting up your infrastructure? If anything, it is poor design on your part. Python is beautiful. I've used it at 3 different companies now, 2 of which i encouraged them to try it out and they have nothing but love for it.

Re: Gluon – Deep Learning API from AWS and Microsoft

#40

Post by saurik »

staticelf wrote: post #3

Can someone please shed light to why so many ML tools and frameworks are being implemented in Python? What makes Python so special for doing ML? Personally, I would love for MS to release or support a .NET based ML toolkit. There is open source stuff like http://accord-framework.net but I would assume that it isn't as big nor complete as a framework being supported by a major corporation.

https://jeffknupp.com/blog/2017/09/15/python-is-the-fastest-...

> Python's Buffer Protocol: The #1 Reason Python Is The Fastest Growing Programming Language Today

> The buffer protocol was (and still is) an extremely low-level API for direct manipulation of memory buffers by other libraries. These are buffers created and used by the interpreter to store certain types of data (initially, primarily "array-like" structures where the type and size of data was known ahead of time) in contiguous memory.

> The primary motivation for providing such an API is to eliminate the need to copy data when only reading, clarify ownership semantics of the buffer, and to store the data in contiguous memory (even in the case of multi-dimensional data structures), where read access is extremely fast. Those "other libraries" that would make use of the API would almost certainly be written in C and highly performance sensitive. The new protocol meant that if I create a NumPy array of ints, other libraries can directly access the underlying memory buffer rather than requiring indirection or, worse, copying of that data before it can be used.

(The italic emphasis was copied from the original article.)

Post reply on HN