> Lisp allows us to scale dramatically and manage a large code base. Wow, really? How big is your company? > Right now, in our core company we have three people, two here in Virginia and one in Mexico City.
Kina Knowledge, using Common Lisp extensively in their document processing stack
11–20 of 62 posts
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#12> Lisp allows us to scale dramatically and manage a large code base. Wow, really? How big is your company? > Right now, in our core company we have three people, two here in Virginia and one in Mexico City.
You know Google's flight scheduling software runs on Borg right ? After they acquired ITA, they have few openings in between, and are yet, apparently, one of the biggest consumers of quotas.
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#13Really interested to play with dlisp if/when it is open sourced!
Have they said whether it's a complete CL implementation or just a subset? I have the impression that it gets asymptotically more difficult as you approach the finish line.
It's really easy to make something that kind-of looks like CL by use of clever smoke and mirrors, and many "Lisp in X" projects do so. See e.g. parenscript's LOOP implementation.
Other notes:
- Implementing CLOS is a massive undertaking, but there are two reasonably good portable implementations of it (Closette and PCL).
- The numeric tower is sufficiently loosely specified that you can typically get away with using any existing bignum library.
- LOOP and FORMAT are annoying to implement just because they are big DSLs. I'm also not aware of any portable implementations of them (though SICL might have one).
- Most of the rest of the effort will be spent on implementing things that interact with the system. Streams/files/pathnames are all annoying
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#14Earlier quoted context omitted.
Have they said whether it's a complete CL implementation or just a subset? I have the impression that it gets asymptotically more difficult as you approach the finish line.
Assuming you aren't just talking about how any project becomes harder to fix bugs in once you've fixed the easy bugs, it only gets more difficult as your approach the finish line if you don't plan on making a CL implementation from the start. It's really easy to make something that kind-of looks like CL by use of clever smoke and mirrors, and many "Lisp in X" projects do so. See e.g. parenscript's LOOP implementation…
Also, if you're willing, it's possible to just lift a bunch of code from SBCL/CMUCL due to permissive license.
MIT LOOP https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/...
few other versions including symbolics update to MIT which is supposedly closer to ANSI: https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/...
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#15> Lisp allows us to scale dramatically and manage a large code base. Wow, really? How big is your company? > Right now, in our core company we have three people, two here in Virginia and one in Mexico City.
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#16This part is really interesting: > It is fast - our spatial classifier takes only milliseconds to come to a conclusion about a page (there is additional time prior to this step due to the OpenCV processing - but not too much) and identify it and doesn’t require expensive hardware. Most of our instances run on ARM-64, which at least at AWS, is 30% or so cheaper than x86-64. The s-expression structures align to documen…
One of the problems we had with promoting commercial use of Scheme and then Racket was that -- although some companies were using it to great success -- there weren't any job postings for it.
It was the norm for a single programmer to be doing the work that would normally be a team (sometimes multiple teams).
And the knowledge of that success wouldn't be well-known. (Because they liked to focus on the work, or because the larger team of business etc. people they were in was also small, or, in at least one case, the business person thought "we use Lisp" would kill business deals even though the code wasn't customer-visible.)
So there would be no success stories, no job postings mentioning it as something people should learn, etc.
Which, I suppose was good for open communities self-limiting themselves to people who were genuine enthusiasts not motivated by money, and with no need to posture as influencers or do SEO, but... not so great for bringing in large developer base, getting lots of startups using it, etc.
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#17> Lisp allows us to scale dramatically and manage a large code base. Wow, really? How big is your company? > Right now, in our core company we have three people, two here in Virginia and one in Mexico City.
oh no no, scale only means hiring tons of programmers after winning funding rounds in these parts
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#18This part is really interesting: > It is fast - our spatial classifier takes only milliseconds to come to a conclusion about a page (there is additional time prior to this step due to the OpenCV processing - but not too much) and identify it and doesn’t require expensive hardware. Most of our instances run on ARM-64, which at least at AWS, is 30% or so cheaper than x86-64. The s-expression structures align to documen…
> The article also mentioned they have 3 programmers One of the problems we had with promoting commercial use of Scheme and then Racket was that -- although some companies were using it to great success -- there weren't any job postings for it. It was the norm for a single programmer to be doing the work that would normally be a team (sometimes multiple teams). And the knowledge of that success wouldn't be well-known…
Be careful with what you conclude from this observation. Is it that A) Lispers are generally 10x Programmers, or that B) Lisps are generally poor languages for team environments?
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#19This part is really interesting: > It is fast - our spatial classifier takes only milliseconds to come to a conclusion about a page (there is additional time prior to this step due to the OpenCV processing - but not too much) and identify it and doesn’t require expensive hardware. Most of our instances run on ARM-64, which at least at AWS, is 30% or so cheaper than x86-64. The s-expression structures align to documen…
> The article also mentioned they have 3 programmers One of the problems we had with promoting commercial use of Scheme and then Racket was that -- although some companies were using it to great success -- there weren't any job postings for it. It was the norm for a single programmer to be doing the work that would normally be a team (sometimes multiple teams). And the knowledge of that success wouldn't be well-known…
One thing that surprised me when I entered the professional world was just how much this is seen as a downside. At almost every level, companies will choose technologies and techniques that allow them to hire more headcount, even if that results in a worse end product.
Many startups value the appearance of having a large actively-hiring engineering team, and many BigCo middle managers want lots of direct reports. They don't usually care how much work gets done per employee, and sometimes they don't even care about the total amount of work that gets done across the organization. It's all about getting warm bodies into seats, either to impress investors or to gain status within the organization.
Re: Kina Knowledge, using Common Lisp extensively in their document processing stack
#20Earlier quoted context omitted.
You know Google's flight scheduling software runs on Borg right ? After they acquired ITA, they have few openings in between, and are yet, apparently, one of the biggest consumers of quotas.
What's a quota?