Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leaf: Machine learning framework in Rust

github.com

Post reply on HN
Print view View original

11–20 of 53 posts

Re: Leaf: Machine learning framework in Rust

#11

Post by [deleted] »

jamesblonde wrote: post #9
ybrah wrote: post #6

Its interesting to see "technical debt" become a more common term. Is there a rigid definition for it? From the article: "Leaf is lean and tries to introduce minimal technical debt to your stack." What exactly does that mean?

It's code that you write (typically quickly), that you know will need to be re-written at a later stage. It's debt that will need to paid at some stage in the future. You didn't do it right first time. Technical debt typically arises because the code was poorly structured or the programmer used the wrong tools/libraries (from a longer-term perspective) or didn't abstract when she should have. The current obsession wi…

[deleted]

Re: Leaf: Machine learning framework in Rust

#12

Post by pmarreck »

ybrah wrote: post #6

Its interesting to see "technical debt" become a more common term. Is there a rigid definition for it? From the article: "Leaf is lean and tries to introduce minimal technical debt to your stack." What exactly does that mean?

Yes. https://en.wikipedia.org/wiki/Technical_debt

I've seen it firsthand. Basically, it's the accumulation of suboptimal code, over time, usually due to time constraints imposed by management. In short, any time you do a dirty hack just to get something working and meet a deadline, and then don't find the time to refactor that code into a working non-hack, you have piled a bunch of manure onto the technical-debt heap. But it also seems to be a side-effect of normal code accretion to a codebase while on a team- in other words, there seems to be no way to avoid it entirely. It's like cancer, in biology. ;)

TD-ridden code is often not modular, not unit-tested, has many dependencies (spaghetti) which are then difficult to remove or replace and tend to trigger cascading bugs/failures, has too many responsibilities, has very long methods/functions, uses mutable state (changes global state which can then impact other parts of the codebase or make concurrency impossible), or is otherwise difficult to maintain.

An example of "working" tech debt is the "God class" in codebases, the model that the entire business depends on but which is over-laden with responsibilities. The risk to change it is too great (due to the business dependence) so it becomes a constant thorn in the side of maintaining the code.

The "debt" part comes from the fact that at some point you are expected to "repay" it (via costly man-hours of refactoring work). The benefit of doing so is potentially multifold, though: Faster/more modular/better-written code, faster tests (and therefore better productivity), better designs in general, more resilient code, more maintainable code, less buggy code, etc. etc.

The only known resolutions of tech debt are costly refactorings or global rewrites. The way to reduce the risk there is to first unit-test the existing code. These books help:

http://smile.amazon.com/Growing-Object-Oriented-Software-Gui...

http://smile.amazon.com/Refactoring-Improving-Design-Existin...

Re: Leaf: Machine learning framework in Rust

#13

Post by mjhirn »

zump wrote: post #2

Any recurrent layers?

We would love to have them and compare their performance with recurrent layers of other frameworks[1]. There exists an issue for the implementation of recurrent layers in Leaf (#73)[2].

[1]: http://autumnai.com/deep-learning-benchmarks.html

[2]: https://github.com/autumnai/leaf/issues/73

Re: Leaf: Machine learning framework in Rust

#14

Post by lloyd-christmas »

choosername wrote: post #10
jamesblonde wrote: post #9

Earlier quoted context omitted.

It's code that you write (typically quickly), that you know will need to be re-written at a later stage. It's debt that will need to paid at some stage in the future. You didn't do it right first time. Technical debt typically arises because the code was poorly structured or the programmer used the wrong tools/libraries (from a longer-term perspective) or didn't abstract when she should have. The current obsession wi…

it could be any kind of maintainance, I thought

My view on it is that I lean towards _known_ future maintenance at the time of programming. Adding a global variable because you aren't sure how it's going to tie in with someone else's current feature is a bit different than adding a global variable because you think it's how it's supposed to be done. I try to make the point of distinguishing bad code and technical debt, as it becomes pretty easy to just say "it's technical debt" as an excuse for doing something poorly. I tend to put general code maintenance in a different bin. To each their own, though.

Re: Leaf: Machine learning framework in Rust

#15

Post by NotUsingLinux »

hobofan wrote: post #7
IshKebab wrote: post #3

I think Microsoft's approach with CNTK is far preferable to this. Rather than defining all the layers in Rust or C++ it uses a DSL to specify mathematical operations as a graph. You can easily add new layer types, and recurrent connections are easy too - you just add a delay node. Furthermore, since the configuration file format is fairly simple, it is possible to make GUI tools to visualise it and - in future - edit…

A DSL based format has some advantages as it easy to get going with building networks. However you are then constrained by what the program that interprets/executes the DSL supports in terms of loading/saving data, solvers etc.. If you want to do something more dynamic e.g. AlphaGo then you have to go back to a "real" programming language anyway. That's not to say that Leaf won't have a DSL at some point, but we will…

so you say prototyping is easier without a DSL?

Re: Leaf: Machine learning framework in Rust

#16

Post by YeGoblynQueenne »

> super-human image recognition

That's a bold claim. As far as I know there was one paper that reported a model beating human scores in a specific test (imagenet, I believe). Whether that translates to "superhuman" results in general is followed by a very big question mark.

In general I really struggle to see how any algorithm that learns from examples, especially one that minimises a measure of error against further examples, can ever have better performance than the entities that actually compiled those examples in the first place (in other words, humans).

I'm saying: how is it possible to learn superhuman performance in anything from examples of mere human performance at the same task? I don't believe in magic.

Re: Leaf: Machine learning framework in Rust

#17

Post by NotUsingLinux »

pmarreck wrote: post #12
ybrah wrote: post #6

Its interesting to see "technical debt" become a more common term. Is there a rigid definition for it? From the article: "Leaf is lean and tries to introduce minimal technical debt to your stack." What exactly does that mean?

Yes. https://en.wikipedia.org/wiki/Technical_debt I've seen it firsthand. Basically, it's the accumulation of suboptimal code, over time, usually due to time constraints imposed by management. In short, any time you do a dirty hack just to get something working and meet a deadline, and then don't find the time to refactor that code into a working non-hack, you have piled a bunch of manure onto the technical-debt heap…

I think this you can view this from a different point of view.

In my opinion this is because what people think software is.

So if you see software as code which expresses what you want, the question is what do you do when it does not do what you want wrong, or do you want something additionally.

So software really is our desire for some specific thing. But it is also a tool which can express arbitrary things. So its a mirror which reflects back on us to discover our real intentions and desires.

Eventually its more of a conversation in which you expand and direct your intentions. And programming or software is just one way to do that.

I think eventually AI will be able to deliver such reflecting conversations to us, the question would be which medium (hardware, operating system, programming language) will it use.

I do not think it will use building blocks (hardware, operating system, programming language) created by humans. Because those are to incomplete and arbitrary.

remember the building blocks allow for plenty of room to allow bootstrapping on multiple levels. An AI could create blocks to create a solution that is so very simple we can't even imagine, yet is unthinkable for humans right now.

Re: Leaf: Machine learning framework in Rust

#18

Post by hobofan »

NotUsingLinux wrote: post #15
hobofan wrote: post #7

Earlier quoted context omitted.

A DSL based format has some advantages as it easy to get going with building networks. However you are then constrained by what the program that interprets/executes the DSL supports in terms of loading/saving data, solvers etc.. If you want to do something more dynamic e.g. AlphaGo then you have to go back to a "real" programming language anyway. That's not to say that Leaf won't have a DSL at some point, but we will…

so you say prototyping is easier without a DSL?

Depends on what kind of prototyping. At the current state of neural networks DSLs are mainly helpful if you want to tune a network architecture for well-established tasks like image classification for the Imagenet dataset.

Outside of that I see more dynamic alternatives used much more.

Re: Leaf: Machine learning framework in Rust

#19

Post by benbou09 »

YeGoblynQueenne wrote: post #16

> super-human image recognition That's a bold claim. As far as I know there was one paper that reported a model beating human scores in a specific test (imagenet, I believe). Whether that translates to "superhuman" results in general is followed by a very big question mark. In general I really struggle to see how any algorithm that learns from examples, especially one that minimises a measure of error against further…

It is much faster than humans

Re: Leaf: Machine learning framework in Rust

#20

Post by wall_words »

The performance graph is deceptive for two reasons: (1) Leaf with CuDNN v3 is a little slower than Torch with CuDNN v3, yet the bar for leaf is positioned to the left of the one for Torch, and (2) there's a bar for Leaf with CuDNN v4, but not for Torch.

It's good to see alternatives to Torch, Theano, and TensorFlow, but it's important to be honest with the benchmarks so that people can make informed decisions about which framework to use.

Post reply on HN