Live data from Hacker News

Make Lisp 15x faster than Python or 4x faster than Java

blog.postabon.com

91–100 of 100 posts

Re: Make Lisp 15x faster than Python or 4x faster than Java

#92
post #43
post #23

Earlier quoted context omitted.

I also wonder why your java benchmark is slow. Java seems to be around 2-3x slower than C on average, this would make your optimized CL to be around 1.5x faster than C.

Define "slower". Languages don't have a fixed "speed" relation between them. Performance varies between machine, language, problem, and implementation technique.

Is any of that so remarkable that we should presume catch23 doesn't know?

A charitable response would help catch23 understand what's different about this one problem from what we can sensibly presume as common experience - not Sun Java, JVM trig functions.

Re: Make Lisp 15x faster than Python or 4x faster than Java

#93
post #52

Isn't it a much better idea to pick a distance comparison algorithm that's not so staggeringly slow to begin with? Why not convert (i.e. pre-compute) the lat/lons to 3-vectors (origin center of the earth) and then compare distances by pythagoras, no fuss, muss or trig and you can even skip the square root. If you need to display an actual accurate distance you can do the (significantly smaller amount of) trig to get…

It's valid if you want to understand what'll happen when you start getting random (lat,long) pairs thrown at you that you can't precompute.

His problem is 'i have a bunch of existing points, find and compare distances to new point'. Doing this by lat,lon and trig is known as one of the slowest ways to do it.

Re: Make Lisp 15x faster than Python or 4x faster than Java

#94
post #93

Earlier quoted context omitted.

It's valid if you want to understand what'll happen when you start getting random (lat,long) pairs thrown at you that you can't precompute.

His problem is 'i have a bunch of existing points, find and compare distances to new point'. Doing this by lat,lon and trig is known as one of the slowest ways to do it.

Um... no. From the article:

"I thought I'd demonstrate this functionality, with a bit of real world code. Postabon recommends deals to users based on a variety of factors such as the age of the deal, other people's votes, your preferences, and (relevant for this example) your distance from the deal. Most of the other factors can be computed asyncronously and just cached, but your distance from deals is computed a lot, and can't really be pre-computed since I have no way of knowing your location a priori (well, that's not strictly true, and we do do some memoization, but it has a relatively low hit rate)."

Re: Make Lisp 15x faster than Python or 4x faster than Java

#95
post #93

Earlier quoted context omitted.

His problem is 'i have a bunch of existing points, find and compare distances to new point'. Doing this by lat,lon and trig is known as one of the slowest ways to do it.

Um... no. From the article: "I thought I'd demonstrate this functionality, with a bit of real world code. Postabon recommends deals to users based on a variety of factors such as the age of the deal, other people's votes, your preferences, and (relevant for this example) your distance from the deal. Most of the other factors can be computed asyncronously and just cached, but your distance from deals is computed a lot…

Yes. He has a bunch of existing points (the deals) and and a new point (user location). Deals are the known points, which can be represented any way you like. lat lon, vectors, cylindrical coords, you name it. The thing not known in advance is the user location. And don't umm... me. It's rude and stupid, especially when you don't seem to know what you're talking about.

Re: Make Lisp 15x faster than Python or 4x faster than Java

#96
post #95

Earlier quoted context omitted.

Um... no. From the article: "I thought I'd demonstrate this functionality, with a bit of real world code. Postabon recommends deals to users based on a variety of factors such as the age of the deal, other people's votes, your preferences, and (relevant for this example) your distance from the deal. Most of the other factors can be computed asyncronously and just cached, but your distance from deals is computed a lot…

Yes. He has a bunch of existing points (the deals) and and a new point (user location). Deals are the known points, which can be represented any way you like. lat lon, vectors, cylindrical coords, you name it. The thing not known in advance is the user location. And don't umm... me. It's rude and stupid, especially when you don't seem to know what you're talking about.

Yup, you're right. I had my concepts in a twist.

Re: Make Lisp 15x faster than Python or 4x faster than Java

#97
post #91

Earlier quoted context omitted.

Of course. There are some benchmarks where Perl is faster than Java 6.

Why haven't you linked to them?

I want to discuss the nature of benchmarks, not which line of Java to change to make one program run one second faster.

Re: Make Lisp 15x faster than Python or 4x faster than Java

#98
post #89

And of course, Haskell can be faster than all of these.

http://gist.github.com/289645 I coded it in haskell, and it is faster, if it's a correct translation. Without parallelizing, it clocks at 1.8 seconds, compiled with $ ghc --make. If anyone plans to compile this you'll need to $ cabal install geodetic, before $ ghc --make filename.hs 1.8 Seconds seems a little too fast though...

It is. You never request the actual calculation to be done on each point, you only request that the last value be calculated. It takes 1.8 seconds to get to the end of the 4-variable cartesian product.

This was my theory, and I tested it by importing Debug.Trace, and then changing the line in question to:

    let result = [traceShow [j,k,l,m] (distance j k l m) | j 
This will print "show [j,k,l,m]" when the actual distance is required. When I run the program, it prints what you'd expect:

    [[-90.0,-180.0,-90.0,-180.0]
    0.0,[-90.0,-180.0,-90.0,-177.5]
    2.2737639212328448e-32,[-90.0,-180.0,-90.0,-175.0]
    9.095050533892612e-32,[-90.0,-180.0,-90.0,-172.5]
    2.046381347867479e-31]
    [90.0,180.0,90.0,180.0]
    0.0
The stuff in brackets is the traceShow output, and it's mixed in with the printing of each result. Lazy evaluation is; it calculates only what is necessary. In this case, it's every (j,k,l,m) quad (so it knows which one is last), and then the 5 values you request.

FWIW, your program runs in 14.28 seconds on my (old) machine, and the strict version runs in 48.94. (This includes a foldl' over the list that ensures every value is evaluated. It may add a bit of overhead.)

Re: Make Lisp 15x faster than Python or 4x faster than Java

#99
post #89

Earlier quoted context omitted.

http://gist.github.com/289645 I coded it in haskell, and it is faster, if it's a correct translation. Without parallelizing, it clocks at 1.8 seconds, compiled with $ ghc --make. If anyone plans to compile this you'll need to $ cabal install geodetic, before $ ghc --make filename.hs 1.8 Seconds seems a little too fast though...

It is. You never request the actual calculation to be done on each point, you only request that the last value be calculated. It takes 1.8 seconds to get to the end of the 4-variable cartesian product. This was my theory, and I tested it by importing Debug.Trace, and then changing the line in question to: let result = [traceShow [j,k,l,m] (distance j k l m) | j This will print "show [j,k,l,m]" when the actual distanc…

great, thanks for the follow up.

Re: Make Lisp 15x faster than Python or 4x faster than Java

#100
post #85
post #21

I ran the Python script through Shedskin, a partial Python implementation that compiles Python to machine code via an intermediate layer of C++. I did not have to declare any types for any variables, or add declarations for speed/safety; Shedskin automatically determined the types. The finished program went down in runtime from 533.7s on my machine, to 24.42 seconds. If we adjust for machine speed (since the author's…

There may be hundreds of man years of research gone into LISP, but a lot of that research found its way into shedskin too. And the Self research is in there too. And his own research (Shedskin was originilly his Masters).

the main techniques that I use for type inference were invented by ole agesen and john plevyak, who worked on self and concurrent aggregates, respectively (iirc). it may be they were influenced by research on lisp, but I doubt it.
Post reply on HN