Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From First Principles: Why Scala?

lihaoyi.com

Post reply on HN
Print view View original

171–180 of 342 posts

Re: From First Principles: Why Scala?

#171

Post by einpoklum »

brabel wrote: post #140
dkarl wrote: post #74

Earlier quoted context omitted.

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

I am fairly certain the jury is out on this question: It's always better to write simple, almost dumb code that anyone can understand than to use advanced abstractions from category theory or whatever. Why? Because every single study or anedocte I've ever heard is like yours: adding complex abstractions to code do NOT make it more reliable. But they do make it much harder to modify, understand, and fix. FP tought us…

> adding complex abstractions to code do NOT make it more reliable.

Sometimes false. If the (more) complex abstractions are in well-tested libraries that haven't been written solely for your code.

> But [complex abstractions] do make [your code] much harder to modify,

Not necessarily. You can separate concerns; you can isolate changes; and you hopefully reduce the amount of code you've written yourself, significantly even.

> [complex abstractions] do make [your code] much harder to understand

This is as likely false as it is true.

> [complex abstractions] do make [your code] much harder to fix

Again, this very much depends. If you're using a well-tested, widely-used external library with those abstractions, it may well be easier to fix your own code.

Re: From First Principles: Why Scala?

#172

Post by dnanabkchsbxb »

dkarl wrote: post #74
dominotw wrote: post #40

Earlier quoted context omitted.

> Hard to win technical arguments with Scala geniuses that like using complicated language features. I was on a team building good old crud apps using monads, monoids, categories, combinators, effects cats, seamless and bunch of other nonsense that i've now purged from my brain. You could've easily mistaken our team for a programming language research group at a university. This is literally the number one reason i w…

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

Reading this thread is giving me some catharsis. I worked on a large project in a big tech company where the org’s leadership was ideologically determined to build everything in Scala. We ended up with dozens of engineers and failed to deliver basic functionality. The leadership eventually left the company and now they are running a startup with a few of the Scala savants from the old team. They got lots of funding and they invited me to visit to get me to join. They showed me the awesome type system they had implemented and tried to demo it but the product was broken in five ways.

Re: From First Principles: Why Scala?

#173

Post by jatcwang »

aynyc wrote: post #15

Is Scala used outside of Spark and Akka? Spark is my company chosen analytic engine, Scala was/is the main language for majority of the production workload. In the last two years or so, more devs are writing PySpark and SparkSQL jobs than Scala.

I've spent the last 5 years working professionally in Scala and I've never touched spark and I know plenty others like me. (I'm mainly a backend developer)

I think it's quite location-dependent. From my own surroundings, you can easily find backend Scala jobs in the bigger hubs in Europe, and seems like there are plenty in the US as well.

Re: From First Principles: Why Scala?

#174

Post by [deleted] »

brainzap wrote: post #85
dominotw wrote: post #40

Earlier quoted context omitted.

> Hard to win technical arguments with Scala geniuses that like using complicated language features. I was on a team building good old crud apps using monads, monoids, categories, combinators, effects cats, seamless and bunch of other nonsense that i've now purged from my brain. You could've easily mistaken our team for a programming language research group at a university. This is literally the number one reason i w…

Clojure still sounds like research group to me.

[deleted]

Re: From First Principles: Why Scala?

#175

Post by sk5t »

adenozine wrote: post #3

If I've said it once I've said it a thousand times: the day Scala kills sbt officially will be the day thousands of devs consider a return to it. Until then, I couldn't dare.

Scala compiles well enough under Gradle--got it in a roughly dozen-module multi-language project right now, and it's no trouble. I suppose Gradle might (or might not?) come up short if you want automatic deprecation/replacement updates applied to source.

Re: From First Principles: Why Scala?

#176

Post by jmfldn »

dkarl wrote: post #74
dominotw wrote: post #40

Earlier quoted context omitted.

> Hard to win technical arguments with Scala geniuses that like using complicated language features. I was on a team building good old crud apps using monads, monoids, categories, combinators, effects cats, seamless and bunch of other nonsense that i've now purged from my brain. You could've easily mistaken our team for a programming language research group at a university. This is literally the number one reason i w…

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

"I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language"

I don't mean to be rude to your colleagues, but almost by implication in what you've said, they're not good programmers. Being a good programmer is nothing to do with leveraging a fancy FP language. It's about delivering working, maintainable, testable code. Whether you write buggy spaghetti code in Go or misuse Scala, its the same root problem.

I've been a Scala dev for five years, worked on some massive projects and it's been a dream. As long as you agree on a style, don't go crazy with the language and use it with discernment, it's a joy to work with. It's a sharp tool though and it takes discernment to know how to wield it appropriately. I don't know if that's a downside, it's more a warning.

Re: From First Principles: Why Scala?

#177

Post by the_only_law »

aynyc wrote: post #15

Is Scala used outside of Spark and Akka? Spark is my company chosen analytic engine, Scala was/is the main language for majority of the production workload. In the last two years or so, more devs are writing PySpark and SparkSQL jobs than Scala.

I want to like Scala and Akka, but they just seem to get used very little in the “real world”. I recall some HN comments complaining about how a lot of the Scala they were writing for Spark was more or less a series of procedural API calls rather than using the language to its potential.

Re: From First Principles: Why Scala?

#178

Post by MikeDelta »

brabel wrote: post #140
dkarl wrote: post #74

Earlier quoted context omitted.

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

I am fairly certain the jury is out on this question: It's always better to write simple, almost dumb code that anyone can understand than to use advanced abstractions from category theory or whatever. Why? Because every single study or anedocte I've ever heard is like yours: adding complex abstractions to code do NOT make it more reliable. But they do make it much harder to modify, understand, and fix. FP tought us…

When I was young I though that good writers are the ones who write such complex sentences that nobody can understand. Later I learned that good writers are clear and understandable.

I used to write complex code because I found it elegant. Now I write simple code because I enjoy breaking down a complex problem into its simplest form, which for me means I fully understand the problem.

Re: From First Principles: Why Scala?

#179

Post by dkarl »

rovolo wrote: post #169
dkarl wrote: post #133

Earlier quoted context omitted.

As I am slowly acquiring the habits of thought that make it possible to read and write FP code at a reasonable speed, I'm horrified by how much idiomatic Scala FP code relies on projecting certain assumptions onto types and operations that seem, at first glance, to be neutral mathematical abstractions. For example, I find myself hating the Either type, because I feel like there is a socially established convention th…

> I often handle Either (and Option) using pattern matching when I feel it's important to give both code paths equal importance and equal visibility in the code, but people change it because flatMap is supposedly more idiomatic, and they believe that eliminating pattern matching from their code is a sign of sophistication. Isn't this the same as letting exceptions bubble up in a non-FP language?

It's supposed to work like that, but it's a lot easier to screw up. Smart people screw it up all the time, and it's hard to spot in code review, whereas average programmers have no problem avoiding swallowing exceptions once they realize it's important, and if they do mess up it stands out like a sore thumb in the code.

Re: From First Principles: Why Scala?

#180

Post by simias »

brabel wrote: post #140
dkarl wrote: post #74

Earlier quoted context omitted.

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

I am fairly certain the jury is out on this question: It's always better to write simple, almost dumb code that anyone can understand than to use advanced abstractions from category theory or whatever. Why? Because every single study or anedocte I've ever heard is like yours: adding complex abstractions to code do NOT make it more reliable. But they do make it much harder to modify, understand, and fix. FP tought us…

Finding the right level of abstraction is basically the one and only skill that matters for a software engineer IMO. I've got about 15 years of experience in programming and it's still something I'm actively working on.

I disagree with the general take that "adding complex abstractions to code do NOT make it more reliable". Good use of abstraction makes code easier to write, easier to maintain and easier to extend. Good use of abstraction can make code more concise and more regular while at the same time allowing better diagnostics when you do something wrong.

As a quick example: a generic JSON serialization library that can work with any type is probably a lot nicer to use that one that requires manual reimplementation for every class in your program. It'll be more complicated to write but it's probably well worth it in the end. Similarly a logging system that can abstract over several backends and log levels depending on the environment probably beats having a bunch of if/else every time you want to log a message.

But one needs to remember the old saying: debugging code is harder than writing it, so if you write code that's as smart as you're capable of producing then by definition you're not smart enough to debug it.

Post reply on HN