Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After 7 years in production, Scarf has reluctantly moved away from Haskell

avi.press

Post reply on HN
Print view View original

51–60 of 320 posts

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#51

Post by em-bee »

rpbiwer2 wrote: post #41
em-bee wrote: post #19

Earlier quoted context omitted.

python has optional types too now. if you could get the LLM to produce typed python would it be any worse than typescript?

Yes, because the code you (or the LLM) writes is only part of the equation; if you use third party libraries then it becomes an ecosystem problem. I'm not currently using much Python but my understanding is that the community has not yet aligned on typing nearly as well as the TS community has.

my understanding is that the [python] community has not yet aligned on typing nearly as well as the TS community has

this is undoubtedly true.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#52

Post by derdi »

> The type system caught real bugs.

> The model can often avoid the mistake before the compiler ever sees the code.

> The type safety we gave up hasn’t been noticeable in any concrete way yet [...]

> Type safety can be a huge advantage for LLM-generated code if the compiler is helping the agent converge quickly.

Well, good to have this question cleared up once and for all :-)

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#53

Post by eduction »

crux wrote: post #25

I strongly agree with the premise of this article, which is why I am surprised that the author moved away from Haskell to Python. For some time now it’s felt clear (or at least extremely) compelling that agents need fast compile times in order to be effective, especially when you’re working in parallel. But the other thing that has felt just as obvious is that agents need strong type systems and narrow guardrails in…

>agents need strong type systems and narrow guardrails

I read the second paragraph of linked article as saying close to the opposite of that, particularly,

"the model can often avoid the mistake before the compiler ever sees the code. And as the models get better, the relative value of catching every possible issue at compile time changes."

In other words, LLMs are much less likely than humans to make dumb, fat-finger mistakes, and, when they do, are able to catch and fix them more quickly, ergo the value of type checking has fallen.

Everything in the prior sentence is, obviously, highly debatable. But it felt like part of the premise.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#54

Post by derdi »

matt2000 wrote: post #5

I am increasingly wondering if we are in a post-language world in terms of development. Why would I ask an agent to write a server in anything other than the most efficient language, although efficiency can take several forms: runtime, token usage during development, and wall clock dev time (affected by slow compile times for example). My intuition is that type-safe languages with fast compilers are the best option.…

My thought reading this article was: Why write the system in any one language at all? And I don't just mean having some parts in one language and others in another language, I mean redundant implementations of the same parts. You can use an AI to rewrite parts of the system, and then throw away the old part... or you could just keep the old part.

That is: Have a Haskell base system. Have a Python "development" version on which you iterate at lightning speed. But also, in the background, moving at whatever pace it takes, have an agent running that imports all the Python development changes into the Haskell version. Have nightly builds of the Haskell version to reap its benefits (issues caught by the type system, more efficient native code). They must have continuous or nightly processes to fix bugs in the Python code anyway, there is no way that all the things they ship "while still on the call with the customer" are always tested on the full test suite and always 100% correct.

And it doesn't have to be Python/Haskell of course. The "development" version could be a (hypothetical?) interpreted Haskell. I have no idea if ghci would be useful for this. Neither do I know if the 15-minute Haskell build time is spent in the frontend (so an interpreter would have to pay that cost too) or in code generation or linking (which the interpreter wouldn't need to care about). Anyway, these are things I would think about before I did what the OP did.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#56

Post by marcosdumay »

muragekibicho wrote: post #3

I'm not trying to be reductive but the article's a lot of words for "We're vibecoding our app now and the glorious (almost almighty) Haskell compiler is too slow for the agent to iterate it's mistakes until it gets it right."

Mixed with some complaints on how the community doesn't like vibe-coding, and that if you insist on not letting AI think for you, you will be left to die in the dust of the other competitors.

The amount of certainty random people have that LLMs have already revolutionized software development seems to be directly proportional to the media awareness of the AI companies finance unsustainability.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#57

Post by reinitctxoffset »

I'm internally dogfooding my take on the stack that makes all these problems go away.

Everything sort of exists, but it's this heinous zero documentation, high pain tolerance thing: buck2 and RBE with NativeLink and hooking that up to action runners and it needs to all work in a container or on nix or in a deb and on MacOS, you hand roll the auth and the certs and where do your compilers come from, can it do NVIDIA, can it do mobile.

Problem is switching off Haskell doesn't help for long: the agents proliferate and you're back where you started with more bugs. So I've been sucking it up and getting all this shit one click and it works. This is good enough for my use, and if the Scarf folks want a solution and are willing to work with a garage band startup, I'd be open to doing a closed alpha. I have a buck2 where you write the rules in Haskell (if you even need to change the prelude, it ships with a WASM that isn't coupled to fbcode), and the Nix cache/substitutor is backed by NativeLink so it scales to anything and it speaks all the protocols correctly and with a verified supply chain.

I'm not even really sure this will become a product, I just need it, but I sort of suspect others will need it too. If there's interest I'll put up a landing page with an email sign up thing.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#58

Post by derdi »

weinzierl wrote: post #4
muragekibicho wrote: post #3

I'm not trying to be reductive but the article's a lot of words for "We're vibecoding our app now and the glorious (almost almighty) Haskell compiler is too slow for the agent to iterate it's mistakes until it gets it right."

This thought completely neglects the idea that Haskell probably needs significantly less compiler runs because every run catches more errors and gives more information about them. And that is not even considering how often the agent needs to run tests to get it right.

It seems pretty clear that they do only minimal live testing during the "open a ticket, implement something, deploy it in production, all while the customer is still on the call" cycles. So your second concern is probably not relevant in this particular setting.

Regarding the first, I think you're probably right, but then again, if there is a 15-minute base cost, it's hard to amortize that through fewer incremental runs of the compiler.

(Which isn't to say that I think they are doing the right thing.)

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#59

Post by em-bee »

eduction wrote: post #53
crux wrote: post #25

I strongly agree with the premise of this article, which is why I am surprised that the author moved away from Haskell to Python. For some time now it’s felt clear (or at least extremely) compelling that agents need fast compile times in order to be effective, especially when you’re working in parallel. But the other thing that has felt just as obvious is that agents need strong type systems and narrow guardrails in…

>agents need strong type systems and narrow guardrails I read the second paragraph of linked article as saying close to the opposite of that, particularly, "the model can often avoid the mistake before the compiler ever sees the code. And as the models get better, the relative value of catching every possible issue at compile time changes." In other words, LLMs are much less likely than humans to make dumb, fat-finge…

i read that too, but i am highly skeptical. i wish the author would investigate that claim and provide some actual examples substantiating it.

Re: After 7 years in production, Scarf has reluctantly moved away from Haskell

#60

Post by zeendo »

I am surprised by this take, honestly.

We're a Haskell shop (and have been for over 10 years now) and are finding agentic development with Haskell to work pretty damn well.

Cold compile times in Haskell are painful indeed. Our development practices don't really cause us to do that much - even with agents.

It's unclear to me if the development practices at Scarf that cause them to hit this pain often are worth it if it means giving up Haskell because the compile times are too bad. Maybe they are, but I don't think so.

Post reply on HN