Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retro on the Julia Programming Language

blog.staffjoy.com

Post reply on HN
Print view View original

21–30 of 32 posts

Re: Retro on the Julia Programming Language

#21

Post by pjmlp »

sgt101 wrote: post #20
pjmlp wrote: post #19

Earlier quoted context omitted.

I wonder how you managed to have public access to Java 0.7, internal Sun releases?

Do you know, I simply don't remember - maybe it was 1.0, but I have a feeling that I downloaded a beta and was overjoyed at being able to scrap all my hoaky c++ sockets / CORBA code and reimplement in something amazingly easy called RMI ! Not sure why I thought 0.7, perhaps I was wishful thinking wrt. how much change happened in early Java, but I think that the conventions of version numbering hadn't settled in the s…

The public release was 1.0, quickly followed by 1.0.1.

I also shared the same feeling, given the chaos of writing portable C++ code back then.

Re: Retro on the Julia Programming Language

#24

Post by philip1209 »

SixSigma wrote: post #22

Hi Philip, You labast the lack of testing tools but one of the other blog posts [1] is about how you built a testing framework. We're just just being kind in that blog post because you didn't really spell out any shortcomings then ? [1]

Testing support is basic and, when we started, there were no clear best practices - so we published how we organized our repo.

The `@test foo` command basically raises an exception if the result is false. You have to manually register all of the tests and call them individually. setUp and tearDown has to be done manually. The testing library keeps no data about how many tests were run - it just throws an exception if an `@test` encounters a false.

It works, but there is no sophistication.

Re: Retro on the Julia Programming Language

#25

Post by idunning »

philip1209 wrote: post #24
SixSigma wrote: post #22

Hi Philip, You labast the lack of testing tools but one of the other blog posts [1] is about how you built a testing framework. We're just just being kind in that blog post because you didn't really spell out any shortcomings then ? [1]

Testing support is basic and, when we started, there were no clear best practices - so we published how we organized our repo. The `@test foo` command basically raises an exception if the result is false. You have to manually register all of the tests and call them individually. setUp and tearDown has to be done manually. The testing library keeps no data about how many tests were run - it just throws an exception if…

This is partially fixed in Julia 0.5 (or available now as BaseTestNext.jl in Julia 0.4). Its basically a backwards-compatible rewrite of the former Base.Test that adds the notion of test sets - enabling counting tests by default, but also not immediately failing out on the first bad test and even logging.

Re: Retro on the Julia Programming Language

#27

Post by idunning »

philip1209 wrote: post #4
dandermotj wrote: post #3

I'm really impressed you used Julia in production. Despite its immaturity and all the reasons you mentioned, building your product on it is a testament to the language, its potential and the core group working on it. I wonder what the performance differences in your product are going from Julia to python? I've had my eye on it for the last two years and really look forward to its 1.0 release. There is no doubt there…

Our Julia library did shift generation and shift assignment in one step. We split these tasks into two separate microservices as we transitioned away from Julia. So, it is tough to compare benchmarks. We are using some compiled libraries in Python that make these calculations parallel. I was playing with Legos recently and thought of a new way to approach the shift decomposition problem, and I'm amid a prototype in G…

Are you using column generation/Dantzig–Wolfe decomposition?

Re: Retro on the Julia Programming Language

#28

Post by tavert »

SixSigma wrote: post #22

Hi Philip, You labast the lack of testing tools but one of the other blog posts [1] is about how you built a testing framework. We're just just being kind in that blog post because you didn't really spell out any shortcomings then ? [1]

[1] = https://blog.staffjoy.com/test-driven-development-in-juliajk... ?

Re: Retro on the Julia Programming Language

#29

Post by philip1209 »

idunning wrote: post #27
philip1209 wrote: post #4

Earlier quoted context omitted.

Our Julia library did shift generation and shift assignment in one step. We split these tasks into two separate microservices as we transitioned away from Julia. So, it is tough to compare benchmarks. We are using some compiled libraries in Python that make these calculations parallel. I was playing with Legos recently and thought of a new way to approach the shift decomposition problem, and I'm amid a prototype in G…

Are you using column generation/Dantzig–Wolfe decomposition?

We're not using column generation in the current algorithms (but Leanform is just launching with column generation on top of CBC - https://angel.co/leanframe ).

We're not using Dantzig-Wolfe decomposition, but our original scheduling algorithm used a dynamic programming approach that was quite similar.

We're currently amid rewriting the Chomp decomposition algorithm to be a more specialized branch and bound variant.

Re: Retro on the Julia Programming Language

#30

Post by sebastienroy »

philip1209 wrote: post #29
idunning wrote: post #27

Earlier quoted context omitted.

Are you using column generation/Dantzig–Wolfe decomposition?

We're not using column generation in the current algorithms (but Leanform is just launching with column generation on top of CBC - https://angel.co/leanframe ). We're not using Dantzig-Wolfe decomposition, but our original scheduling algorithm used a dynamic programming approach that was quite similar. We're currently amid rewriting the Chomp decomposition algorithm to be a more specialized branch and bound variant.

Yes do do have a column generation algorithm that leverage my expertise I gained working at ad-opt (airline pilot rostering).

We also migrated from julia to python. Another vote for the MIT teams that worked on the project! It really helped us at the beginning!

We are also playing with the google glop solver that gives pretty good results so far.

The main drawback of python for column generation (and as far as I understand, the same flaw exists in juliaopt/JuMP) is twofold. 1) most libraries are "math formulation oriented" which makes implementation hard to maintain/debug 2) biggest issue: the wrappers around the underlying solver (cbc, glpk, etc) keeps all references (to variables and coefficients) and do not offer a clean add/remove interface (only add/set bounds). Even playing with coefficients isn't always straigthforward.

So we had to build our own wrapper on top of the LP wrappers out there (Pulp, pyomo, cylp etc). When solving large scale problems with column generation, you generate hundreds of thousands of different variables even if only a fraction of them are "active" at a time leading to memory and runtime leaks in the usual wrappers. We had cases where the LP library was spending about 85% of the time just writing and reading files for CBC. The or-tools library wraps the C library itself via SWIG and is our best horse as of writing this.

For sure we will think of rolling our own binder too. Last year I was hoping to do it is scheme :) We might get "stuck" with python for the time being though.

Post reply on HN