Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Lisp in Production

tech.grammarly.com

Post reply on HN
Print view View original

21–30 of 119 posts

Re: Running Lisp in Production

#21

Post by lambdaelite »

lispm wrote: post #10
BlanketLogic wrote: post #2

Very informative. Thank you. Anyone here has any experience with the GCs of Allegro or LispWorks or any other commercial Lisp implementations?

LispWorks has a lot of features in its GC. Years ago it was used in a demanding telco application, an ATM switch. It was also used on a space mission experiment. Generally the runtime is very very nice. Franz Inc uses Allegro CL in a large database. They tuned the GC quite a bit for that. But there were also other GC demanding applications on Allegro CL, for example in CAD and 3D design. They are now working on a con…

Why are concurrent GCs rare?

Re: Running Lisp in Production

#22

Post by jeremiep »

apalmer wrote: post #15
lkrubner wrote: post #11

Good lord, I would go insane if I ran into a bug like this: "We've built an esoteric application (even by Lisp standards), and in the process have hit some limits of our platform. One unexpected thing was heap exhaustion during compilation. We rely heavily on macros, and some of the largest ones expand into thousands of lines of low-level code. It turned out that SBCL compiler implements a lot of optimizations that a…

I dont really understand the downvotes... I agree with you would go nuts. My two cents, if your macro is expanding to thousands of lines of code, your doing something wrong I think. I would expect Macros to expand out to a few lines of code which might have function calls that themselves may contain however many lines of code... but expanding to thousands of lines INLINE via macros seems wrong.

Some macros expand to a lot of code, especially when doing more in-depth transformations such as those performed by core.async in Clojure where they transform standard sequential code into a state machine with exactly the same semantic but with the added ability to execute, yield and resume like a coroutine.

Re: Running Lisp in Production

#23

Post by vseloved »

jon-wood wrote: post #14

Apparently they use "JVM languages", JavaScript, Python, Go, Lisp and Erlang in production. I may be in the minority, but that would drive me mad. I assume they're not routinely jumping between those stacks multiple times a day, but even so is there really that much benefit that it's worth keeping track of how to do things in that many different environments?

As for me, adaptability is one of the important traits of a senior engineer. Surely, you don't have to be an expert in every platform, but you also shouldn't go mad if you need to do some work outside of your comfort zone occasionally. Besides, every language has its strong and weak points: if you're putting arbitrary limits here, you're just limiting what you can do and the people you're going to get in a team. At Grammarly, we always erred on the side of more freedom and it worked not so bad for us so far. Although, there are different companies, each with a unique story...

Re: Running Lisp in Production

#24

Post by davelnewton »

jon-wood wrote: post #14

Apparently they use "JVM languages", JavaScript, Python, Go, Lisp and Erlang in production. I may be in the minority, but that would drive me mad. I assume they're not routinely jumping between those stacks multiple times a day, but even so is there really that much benefit that it's worth keeping track of how to do things in that many different environments?

While a Single Language to Rule Them All would be cool, I ultimately prefer using the "best" language for the job based on specific requirements.

The "best" might change over time, too.

It can be a headache to manage massively-polyglot environments. At the same time, it's also pretty great for a variety of reasons. I mean, we regularly use different data stores, messaging solutions, frameworks, etc. and I don't see why languages shouldn't also be up for shuffling.

Re: Running Lisp in Production

#25

Post by BlanketLogic »

ChargingWookie wrote: post #7
BlanketLogic wrote: post #2

Very informative. Thank you. Anyone here has any experience with the GCs of Allegro or LispWorks or any other commercial Lisp implementations?

I've worked a lot with LispWorks and tuning the gc had the same method of programmatically calling a full GC after every N operations. We also found setting the gc threshold to high amount helped a bunch. Supposedly they have a concurrent GC in the works but I haven't played with it.

Thanks.

> I've worked a lot with LispWorks and tuning the gc had the same method of programmatically calling a full GC after every N operations.

What is "full GC", here? Do you mean, even the "older" generations? (Assuming Lispworks also has generational GC a la sbcl)

In other words, Would it have helped if the implementation was a mark-compact rather than generational?

Re: Running Lisp in Production

#26

Post by mrottenkolber »

apalmer wrote: post #15
lkrubner wrote: post #11

Good lord, I would go insane if I ran into a bug like this: "We've built an esoteric application (even by Lisp standards), and in the process have hit some limits of our platform. One unexpected thing was heap exhaustion during compilation. We rely heavily on macros, and some of the largest ones expand into thousands of lines of low-level code. It turned out that SBCL compiler implements a lot of optimizations that a…

I dont really understand the downvotes... I agree with you would go nuts. My two cents, if your macro is expanding to thousands of lines of code, your doing something wrong I think. I would expect Macros to expand out to a few lines of code which might have function calls that themselves may contain however many lines of code... but expanding to thousands of lines INLINE via macros seems wrong.

I assume their macros do compilation of an expressive DSL to efficient low level code, as one would expect in their domain. So keep in mind that this is something pretty fancy you can pretty much only do with Lisp (or by writing a full blown compiler).

Re: Running Lisp in Production

#27

Post by ChargingWookie »

BlanketLogic wrote: post #25
ChargingWookie wrote: post #7

Earlier quoted context omitted.

I've worked a lot with LispWorks and tuning the gc had the same method of programmatically calling a full GC after every N operations. We also found setting the gc threshold to high amount helped a bunch. Supposedly they have a concurrent GC in the works but I haven't played with it.

Thanks. > I've worked a lot with LispWorks and tuning the gc had the same method of programmatically calling a full GC after every N operations. What is "full GC", here? Do you mean, even the "older" generations? (Assuming Lispworks also has generational GC a la sbcl) In other words, Would it have helped if the implementation was a mark-compact rather than generational?

We normally used

http://www.lispworks.com/documentation/lw60/LW/html/lw-712.h...

with the full set to nil.

Re: Running Lisp in Production

#28

Post by Homunculiheaded »

jon-wood wrote: post #14

Apparently they use "JVM languages", JavaScript, Python, Go, Lisp and Erlang in production. I may be in the minority, but that would drive me mad. I assume they're not routinely jumping between those stacks multiple times a day, but even so is there really that much benefit that it's worth keeping track of how to do things in that many different environments?

I think SICP states it best:

"In our study of program design, we have seen that expert programmers control the complexity of their designs with the same general techniques used by designers of all complex systems. They combine primitive elements to form compound objects, they abstract compound objects to form higher-level building blocks, and they preserve modularity by adopting appropriate large-scale views of system structure. In illustrating these techniques, we have used Lisp as a language for describing processes and for constructing computational data objects and processes to model complex phenomena in the real world. However, as we confront increasingly complex problems, we will find that Lisp, or indeed any fixed programming language, is not sufficient for our needs. We must constantly turn to new languages in order to express our ideas more effectively."

At a certain level of software design tying together different programming languages that are each the right tool for their job becomes just another type of programming. I currently do data science work, but even then in a given week I typically use R, Python, Lua and Java (and often Scheme in the evenings for fun). Trying to make any one of those languages do something the other is much better at is a phenomenal waste of time.

On the system level, once prototyping ends, if there's something that Java does phenomenally better than R, but we need both, that implies that you have two parts of the system different enough that they shouldn't be tightly coupled anyway. If you write a deep learning algorithm in Lua, but want to do some statisitical analysis on the results of that in R, it's good to force these things to be separated because if in 5 years you find a better model for the Lua part (maybe some better algorithm in Julia) you want to be able to swap it out anyway.

Re: Running Lisp in Production

#29

Post by 0xCMP »

lambdaelite wrote: post #21
lispm wrote: post #10

Earlier quoted context omitted.

LispWorks has a lot of features in its GC. Years ago it was used in a demanding telco application, an ATM switch. It was also used on a space mission experiment. Generally the runtime is very very nice. Franz Inc uses Allegro CL in a large database. They tuned the GC quite a bit for that. But there were also other GC demanding applications on Allegro CL, for example in CAD and 3D design. They are now working on a con…

Why are concurrent GCs rare?

I am guessing[1] that GCs are easier to code correctly without the concurrency and that a GC language is already expected to be slower so it doesn't make sense to do a concurrent GC. Also possibly, the language doesn't support concurrency well. Like a Python or Ruby.

[1] just an educated guess. I have no real knowledge of GCs other than skimming how they work in articles and runtime/language docs.

Re: Running Lisp in Production

#30

Post by krvss »

jessaustin wrote: post #18
jon-wood wrote: post #14

Apparently they use "JVM languages", JavaScript, Python, Go, Lisp and Erlang in production. I may be in the minority, but that would drive me mad. I assume they're not routinely jumping between those stacks multiple times a day, but even so is there really that much benefit that it's worth keeping track of how to do things in that many different environments?

Since "proper service encapsulation" is mentioned, it may be that each team uses whatever they like, and as long as your component speaks http you don't have to look at what other components are doing.

This is exactly how it is (except sometimes it is not HTTP but message queuing etc)
Post reply on HN