Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New Lisp-Stat Release

lisp-stat.dev

Post reply on HN
Print view View original

31–40 of 59 posts

Re: New Lisp-Stat Release

#31

Post by Lisp-Stat »

tpejrl23432 wrote: post #16
lukego wrote: post #11

Earlier quoted context omitted.

I migrated from Lisp to Julia for the ecosystem. It hasn't been worth it from my point of view. I'll migrate back to Lisp eventually. So I can easily imagine packages like this becoming widely adopted /within/ the Lisp community.

Could you elaborate on this a bit ? Lisp has next to no numerical libraries IIRC ?

I think this depends on what part of the statistics universe you're working in.

For example, within Lisp-Stat the statistics routines [1] were written by an econometrician working for the Austrian government (Julia folks might know him - Tamas Papp). It would not be exaggerating to say his job depending on it. These are state of the art, high performance algorithms, equal to anything available in R or Python. So, if you're doing econometrics, or something related, everything you need is already there in the tin.

For machine learning, there's CLML [2], developed by NTT. This is the largest telco in Japan, equivalent to ATT in the USA. As well, there is MGL [3], used to win the Higgs Boson challenge a few years back. Both actively maintained.

For linear algebra, MagicCL was mention elsewhere in the thread. My favourite is MGL-MAT [4], also by the author of MGL. This supports both BLAS and CUBLAS (CUDA for GPUs) for solutions.

Finally, there's the XLISP-STAT archive [5]. Prior to Luke Tierney, the author of XLISP-Stat joining the core R team, XLISP-STAT was the dominate statistical computing platform. There's heaps of stuff in the archive, most at least as good as what's in base R, that could be ported to Lisp-Stat.

Common Lisp is a viable platform for statistics and machine learning. It isn't (yet) quite as well organised as R or Python, but it's all there.

[1] https://github.com/Lisp-Stat/numerical-utilities/blob/master...

[2] https://github.com/mmaul/clml

[3] https://github.com/melisgl/mgl

[4] https://github.com/melisgl/mgl-mat

[5] https://lisp-stat.dev/docs/resources/xlisp/

Re: New Lisp-Stat Release

#32

Post by na85 »

piethesailorman wrote: post #25
_ofdw wrote: post #7

I've been using lisp-stat in production as part of an algorithmic trading application I wrote. It's been very solid, and though the plotting is (perhaps was, in light of this new release) kinda unwieldy, I really enjoyed using it. Excited to check out the newest release.

Did you write that trading program from scratch? Or by chance did you start with: https://github.com/wzrdsappr/trading-core I have been considering giving this code base a spin.

I wrote it from scratch.

Re: New Lisp-Stat Release

#33

Post by na85 »

medo-bear wrote: post #15
_ofdw wrote: post #7

I've been using lisp-stat in production as part of an algorithmic trading application I wrote. It's been very solid, and though the plotting is (perhaps was, in light of this new release) kinda unwieldy, I really enjoyed using it. Excited to check out the newest release.

institutional or retail? i wanted to do this on a retail scale but i don't know where to start as far as APIs for orders and market feeds. do you have any pointers?

Retail.

I do low-freq stat arb and it's modestly profitable. I just use IBKR's API, their snapshot data, and postgres. Nothing fancy.

Feel free to shoot me an email, it's in my profile.

Re: New Lisp-Stat Release

#34

Post by p1esk »

medo-bear wrote: post #27
tpejrl23432 wrote: post #24

Earlier quoted context omitted.

You can hardly compare any of these one-off projects to something like numpy (let alone something like pytorch etc. which Julia is going after).

what do you mean a one-off project? some of these have corporate/institutional backing but thats not the point. if you know how to implement machine learning models then there should be nothing (except maybe availability of pretrained models) preventing you from using common lisp. if you dont want to dig deep and are happy using just the front end (perfectly acceptable) then python is a much better bet all the needed…

Do any of those libraries linked above provide an example of training a resnet or bert on a GPU? I’m not looking for anything fancy, or production quality, just a basic example.

Re: New Lisp-Stat Release

#35

Post by yakubin »

Am I understanding it right that those plots are only generated in HTML form? I.e. I can't use it to generate an EPS file to be included in a LaTeX document?

Re: New Lisp-Stat Release

#37

Post by medo-bear »

p1esk wrote: post #34
medo-bear wrote: post #27

Earlier quoted context omitted.

what do you mean a one-off project? some of these have corporate/institutional backing but thats not the point. if you know how to implement machine learning models then there should be nothing (except maybe availability of pretrained models) preventing you from using common lisp. if you dont want to dig deep and are happy using just the front end (perfectly acceptable) then python is a much better bet all the needed…

Do any of those libraries linked above provide an example of training a resnet or bert on a GPU? I’m not looking for anything fancy, or production quality, just a basic example.

although not necessarily bert or resnet the following probably has all the ingredients for what you are looking for. the author of this library is a research scientist at deepmind since 2015

https://github.com/melisgl/mgl#x-28MGL-BP-3A-40MGL-BP-20MGL-...

Re: New Lisp-Stat Release

#38

Post by p1esk »

medo-bear wrote: post #37
p1esk wrote: post #34

Earlier quoted context omitted.

Do any of those libraries linked above provide an example of training a resnet or bert on a GPU? I’m not looking for anything fancy, or production quality, just a basic example.

although not necessarily bert or resnet the following probably has all the ingredients for what you are looking for. the author of this library is a research scientist at deepmind since 2015 https://github.com/melisgl/mgl#x-28MGL-BP-3A-40MGL-BP-20MGL-...

There's not even a convolutional layer provided as a building block. This project looks pretty dead to me: most of the code is 8 years old, and the last code-changing commit is over 2 years old. I guess it's okay as a proof of concept for ML with Lisp, but I don't think anyone interested in ML would actually consider using it.

Re: New Lisp-Stat Release

#39

Post by peatmoss »

yakubin wrote: post #35

Am I understanding it right that those plots are only generated in HTML form? I.e. I can't use it to generate an EPS file to be included in a LaTeX document?

I believe the engine vega-lite can create an svg, which is pretty easily converted to eps/pdf etc.

Re: New Lisp-Stat Release

#40

Post by medo-bear »

p1esk wrote: post #38
medo-bear wrote: post #37

Earlier quoted context omitted.

although not necessarily bert or resnet the following probably has all the ingredients for what you are looking for. the author of this library is a research scientist at deepmind since 2015 https://github.com/melisgl/mgl#x-28MGL-BP-3A-40MGL-BP-20MGL-...

There's not even a convolutional layer provided as a building block. This project looks pretty dead to me: most of the code is 8 years old, and the last code-changing commit is over 2 years old. I guess it's okay as a proof of concept for ML with Lisp, but I don't think anyone interested in ML would actually consider using it.

i work in ml and you asked for an example so i thought a proof of concept answer is sufficient. as i said somewhere else if you are looking for a ready made framework then this is not for you and should probably stick with pytorch, tf, and maybe even pyro. but if you are willing to engineer your own system for optimal performance while avoiding c++ then you would be served good by common lisp. common lisp for ml is suited for software engineers doing ml not so much statisticians
Post reply on HN