Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After 7 years in production, Scarf has reluctantly moved away from Haskell

avi.press

Post reply on HN
Print view View original

131–140 of 320 posts

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#131

Post by threethirtytwo »

noelwelsh wrote: post #2

Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decre…

You’d be surprised. It works quite well without the static guard rails. But the static guard rails do improve things but not in some extremely obvious way.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#132

Post by cubefox »

andersmurphy wrote: post #118
aviaviavi wrote: post #62

Earlier quoted context omitted.

The number of compiler runs doesn't matter as much as the total elapsed time it takes to finish the task. In just about every test we ran, LLMs are faster at building in Python than Haskell.

Lisp/smalltalk programmers have been going on about this tradeoff for a long time. It mattered before LLMs too. Lisp/Clojure repl allowing you to compile tiny parts of your program inside your running program is incredible for your feedback/iteration loop. Ironically, this is also what makes them shine with LLMs, the LLM has access to the running program and can modify it while it's running to get feedback instantly.…

> I say this as someone who's first programming language was Haskell.

Amazing. How did that happen? Is it true that functional programming is only counterintuitive because almost everyone starts out with an imperative language?

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#133

Post by threethirtytwo »

ErroneousBosh wrote: post #34
noelwelsh wrote: post #2

Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decre…

> I can't imagine using a language without a good type system to catch all the junk the LLM produces One approach would be to not use LLMs.

Another approach is career suicide. Both moves are isomorphic in many companies today and will be pretty much all companies in the future.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#134

Post by kazinator »

I feel there is something missing here. The narrative here suggests that the author's LLMs are generating perfect Haskell code that just needs to be rubber stamped by the compiler for execution. But, oh, that takes too long.

I'm skeptical; I would think that the problem would actually be that mistakes in a large body of Haskell code would be difficult to fix: that massaging the generated slop into compiling ranges from unpleasantly time consuming to intractable.

Might the author be hiding the honest statement of the problem: that he would rather move fast and break things as a slop artist, but the guard rails are too rigid in Haskell?

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#135

Post by pmontra »

hunterpayne wrote: post #93
em-bee wrote: post #8

Earlier quoted context omitted.

exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent. so does this mean that the LLM writes code that is so good that the compiler does not find any more errors? or is it due to the nature of haskell that makes it hard to write bad code to begin with? or just that because the haskell compiler catch…

"proving code correct has been a research topic at some point." It has been an area of active research for 40 years. But almost all the research returned the null result, meaning that the program proving didn't improve code quality (basically it didn't work). Yet somehow a group of programmers, usually fresh out of academia falls for program proving each generation. Strong types do really help but you need a good com…

The issue with formal proofs is that we might end up with correct code that does the wrong thing. I mean, something that the market doesn't care about. At the same time a buggy set of PHP scripts does what people care about and captures the market. Think about 20+ years ago and you find a lot of examples.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#136

Post by antonvs »

noelwelsh wrote: post #2

Wow. Not a Haskell user, but a big user of other languages with expressive type systems (mostly Scala; some Rust). My experience is the complete opposite. I can't imagine using a language without a good type system to catch all the junk the LLM produces. In fact I thought people would move away from languages from poor type systems, like Python, given the cost of using languages with expressive type systems has decre…

> "At Scarf, we started doing all new API work in Python."

Start the countdown timer for how long it takes them to discover that was a mistake.

Nothing to do with Haskell, but good grief, LLMs do not in any way, shape or form save you from the deep, unfixable problems with Python.

At the very least you need all the static checking machinery like Ruff, Pyright, and hefty unit tests that take the place of typechecking if you don't want obvious failures to only show up in production.

I had this recently with an ML training pipeline, where Python is essentially forced on us. A dynamic error occurred after 17 hours of training - something that a real type system could have easily caught.

The solution that the LLM came up to prevent this in future was a complicated Enum-based system that just made me wish I could use a real programming language.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#137

Post by antonvs »

tonyarkles wrote: post #76
em-bee wrote: post #8

Earlier quoted context omitted.

exactly, i find the article a wierd take. i would have thougt that being able to catch errors at compile time is the assurance that the LLM generated code is actually decent. so does this mean that the LLM writes code that is so good that the compiler does not find any more errors? or is it due to the nature of haskell that makes it hard to write bad code to begin with? or just that because the haskell compiler catch…

> what's the solution here? speeding up the haskell compiler? if that were easy, would it not already have happened? I suspect you’ve nailed the answer: it’s probably not easy, although it’s also possible that it just hasn’t ever had a lot of attention paid to it because it’s been generally fast enough for their user base?

Part of the issue is probably that Haskell build performance is perfectly fine for local development, even on rather large systems.

But in commercial production environments, CI pipelines tend to want to build everything from scratch every time, and that slows everything down. Rust has the same issue. Both languages, by default, compile all their dependencies from source, rather than obtaining precompiled artifacts from a repo the way some languages (like Java) do. And their compilers are slower than e.g. Go's. As the article mentions, various kinds of caching can help with that, but that's extra stuff you have to manage and deal with.

I'm not sure this is a bad thing, though. Haskell co-creator Simon Peyton-Jones coined the unofficial Haskell motto, "avoid success at all costs". I tend to agree with that. It would be difficult for Haskell to maintain its conceptual edge if it were a mainstream commercial language.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#138

Post by antonvs »

timcobb wrote: post #91
japgolly wrote: post #88

Earlier quoted context omitted.

It's about the feedback loop being so slow. Agents often compile and run tests to verify their work

Right, they run tests too. A compiler is like a quick test before tests. How are you going to cut out that check and let the LLM "write it faster" is beyond me. The compiler catches errors across codebases that today's LLM can't economically or reliably put into context to perform similar checks. They're totally different tools, today. Also, you can just compile less frequently. But hey, if LLMs are what drove this p…

> But hey, if LLMs are what drove this person from Haskell to Lisp then all the power to them!

I didn't see Lisp mentioned in the article. They moved to Python. Which is certainly a choice.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#139

Post by pmontra »

eduction wrote: post #53
crux wrote: post #25

I strongly agree with the premise of this article, which is why I am surprised that the author moved away from Haskell to Python. For some time now it’s felt clear (or at least extremely) compelling that agents need fast compile times in order to be effective, especially when you’re working in parallel. But the other thing that has felt just as obvious is that agents need strong type systems and narrow guardrails in…

>agents need strong type systems and narrow guardrails I read the second paragraph of linked article as saying close to the opposite of that, particularly, "the model can often avoid the mistake before the compiler ever sees the code. And as the models get better, the relative value of catching every possible issue at compile time changes." In other words, LLMs are much less likely than humans to make dumb, fat-finge…

My experience with Claude and Ruby, Python, Javascript is similar: it's pretty good at finding the array of strings that was passed to a method instead of an array of integers. Think about record ids coming from a JSON API call. Or the single value instead of an array. I don't remember which Python XML parser is fond of returning one or the other according to the cardinality of the sub elements. Anyway, not only it writes the code to handle those cases but it traces the code and it finds the bugs. So type checking at coding time and who cares about writing the type annotations. They would be probably good to speed up the code at runtime but none of my customers use them and none of them is concerned about the current response time of their systems.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#140

Post by lelanthran »

aviaviavi wrote: post #62
weinzierl wrote: post #4

Earlier quoted context omitted.

This thought completely neglects the idea that Haskell probably needs significantly less compiler runs because every run catches more errors and gives more information about them. And that is not even considering how often the agent needs to run tests to get it right.

The number of compiler runs doesn't matter as much as the total elapsed time it takes to finish the task. In just about every test we ran, LLMs are faster at building in Python than Haskell.

> The number of compiler runs doesn't matter as much as the total elapsed time it takes to finish the task. In just about every test we ran, LLMs are faster at building in Python than Haskell.

How much faster? IOW, what's the difference (in minutes/seconds, not in percentages) between vibing Haskell and vibing Python?

Post reply on HN