Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From First Principles: Why Scala?

lihaoyi.com

Post reply on HN
Print view View original

211–220 of 342 posts

Re: From First Principles: Why Scala?

#211

Post by marcinzm »

hocuspocus wrote: post #195
marcinzm wrote: post #185

Earlier quoted context omitted.

> Yep, the Spark codebase is a great example of what a big Scala codebase should look like. A lot of the hardcore Scala community hates Spark which sort of summarizes the situation.

This is a false dichotomy. You'll run into all sorts of pain and frustration as soon as Spark touches your codebase, no matter what kind of Scala you're writing.

I've almost every big data processing system out there and Spark causes no more pain than any other. Less than most. It's also a data processing system not a library so if you're integrating it into existing code (rather than writing code for it and running code on top of it) I'd argue you're doing it wrong.

Re: From First Principles: Why Scala?

#212

Post by jjav »

watwut wrote: post #196
jgwil2 wrote: post #184

Earlier quoted context omitted.

I think people can be good programmers and poor software architects at the same time. Sounds like that's what has happened in this case: brilliant abstractions that are misused and result in a buggy product and less productive team.

Frankly, they are bad programmers. They are smart highly intelligent people who don't have either knowledge or aptitude or willingness to be good programmers. My pet peeve in this business are supposedly good programmers who get praised despite never having actual results. And it is not like they would be rare.

Unfortunately the leetcode whiteboard interviewing strongly encourages this type. Even almost requires it. If all energy is spent on that, it is time not spent learning how to build stable products which can be maintained long term.

Because turns out that popping out algorithm exercise code after another into source control doesn't produce a good product. But that's what current interview fad measures, thus it's what it produces.

Re: From First Principles: Why Scala?

#213

Post by sireat »

rubyn00bie wrote: post #110

I loved Scala until a minor version updated somewhere around 2.9-2.11 destroyed all my hope. Eight years later I still refused to use it to build anything. Yet, with that said, IMHO, it can be fucking great _second_ language. If you haven't used a statically typed language before, i.e. you're from Ruby, JS, Python, et. al then it's one bangin' ass brain rodeo. I'd highly recommend spending a couple months trying to l…

I actually like Scala.

I teach it to complete beginners to programming and generally it goes great - there is a huge demand for junior Scala developers in my neck of woods.

However when I found that 22 argument monster a while ago it was a bit of a wtf moment to me.

Why 22? Why not 21 or 23 ? Is it one of those no one is going to need more than 22 arguments?

Re: From First Principles: Why Scala?

#214

Post by tanin »

I'm a big fan of Scala, though I think the sweet spot for Scala is for a small team with pre-existing Scala's expertise. For a one-person project, I definitely use Scala. I used to use Ruby a lot.

You can move fast with safety with the caveat that the code quality is harder to manage/control.

Contrasting this with golang where they really take the guard rails to the other extreme (e.g. can't compile if a variable is unused, not even compiling just to test it out locally).

Another weak-point of Scala is that the learning curve is steep, especially when it involves SBT, the defacto build system for Scala.

Re: From First Principles: Why Scala?

#215

Post by jjav »

dominotw wrote: post #40
MrPowers wrote: post #36

I am a Scala programmer & think it's a great language. Here are some arguments for why not Scala: * Li's libs (os-lib, upickle, utest) have clean public interfaces, but most Scala ecosystem libs are hard to use, see the JSON alternatives for examples: https://www.lihaoyi.com/post/uJsonfastflexibleandintuitiveJS... * The Mill build tool looks a lot better than SBT, but seems like everyone is still using SBT * Scala mi…

> Hard to win technical arguments with Scala geniuses that like using complicated language features. I was on a team building good old crud apps using monads, monoids, categories, combinators, effects cats, seamless and bunch of other nonsense that i've now purged from my brain. You could've easily mistaken our team for a programming language research group at a university. This is literally the number one reason i w…

> You could've easily mistaken our team for a programming language research group at a university.

I had the misfortune of working with basically this same team, on scala (of course).

I argued for maintainable simple technology but scala was too hip to pass, for many.

Still remember one meeting trying to decipher a bug and while reviewing the code in question the lead scala fan said "It is not reasonable to expect to understand what code does by looking at it. I'll need to go research this for a few days."

I hope to never see scala (or its ilk) ever again. Give me the most stable language and environment, where every question is a FAQ and every odd behavior is documented in every book and I never need to fall into the rabbit hole of language traps. Then I can just work on building the product, which is the whole point.

Re: From First Principles: Why Scala?

#216

Post by asimpletune »

chaorace wrote: post #28
asimpletune wrote: post #19

Earlier quoted context omitted.

Local mutable state as in a var in a function or how did you mean?

Basically that, yes. The idea is, as long as the mutable state starts and ends within the confines of an individual function, you can get away with treating that function as if it were "truly" FP. The main argument against this is that it's a concession that makes the codebase less coherent. The more hardline argument being that the encapsulated mutable state is still more prone to bugs and should be almost always be…

Yeah, this is totally fine.

It’s rare that you need to use vars in Scala, but still if it’s scoped to the function then you know you can mentally discount it once the function terminates.

Re: From First Principles: Why Scala?

#217

Post by asimpletune »

ebruchez wrote: post #134
asimpletune wrote: post #18

Earlier quoted context omitted.

I wish swift were like Scala. It attempts to look and feel like it, but there’s one huge aspect of Scala that is missing in swift that basically makes swift a world apart, which is expressions vs statements. In Scala everything is an expression, meaning everything evaluates to some result even if that result is void. In swift everything is a statement. There’s a big difference because statements don’t return things,…

> In swift everything is a statement. There’s a big difference because statements don’t return things, only functions do. That sounds absolutely awful.

It’s such a small, non-flashy language feature, that honestly matters much more than almost everything else.

Re: From First Principles: Why Scala?

#218

Post by rbonvall »

fmakunbound wrote: post #128
rubyn00bie wrote: post #110

I loved Scala until a minor version updated somewhere around 2.9-2.11 destroyed all my hope. Eight years later I still refused to use it to build anything. Yet, with that said, IMHO, it can be fucking great _second_ language. If you haven't used a statically typed language before, i.e. you're from Ruby, JS, Python, et. al then it's one bangin' ass brain rodeo. I'd highly recommend spending a couple months trying to l…

> only to find this bad boi: https://www.scala-lang.org/api/current/scala/Function22.html Dang. I just checked in the Common Lisp I use and it's a bit higher. What's the reason for having to have at least 22 concrete function signatures like that? CL-USER> call-arguments-limit 4611686018427387903 (62 bits, #x3FFFFFFFFFFFFFFF)

Originally there was no way of abstracting over an arbitrary number of type parameters in the language, so a different trait was required for each arity. The choice of 22 was arbitrary, I think.

This is no longer true in Scala 3, so there will be no such limit: http://dotty.epfl.ch/docs/reference/dropped-features/limit22...

Re: From First Principles: Why Scala?

#219

Post by hocuspocus »

marcinzm wrote: post #211
hocuspocus wrote: post #195

Earlier quoted context omitted.

This is a false dichotomy. You'll run into all sorts of pain and frustration as soon as Spark touches your codebase, no matter what kind of Scala you're writing.

I've almost every big data processing system out there and Spark causes no more pain than any other. Less than most. It's also a data processing system not a library so if you're integrating it into existing code (rather than writing code for it and running code on top of it) I'd argue you're doing it wrong.

One big reason, maybe the biggest, to write Spark jobs in Scala and (not move to pyspark) is code reuse between different components. My team maintains a fairly sizeable library that is common to our Spark jobs and several web services. Spark is a library dependency if you do anything remotely complex with it and it can easily creep up everywhere if you aren't careful.

Decoupling modules isn't always obvious in a codebase that's grown organically for 6-7 years now (long before I joined) and the cohabitation with Spark is inevitably going to cause some pains. A couple examples:

- Play-json codecs aren't serializable (ironically enough, Circe, the "hardcore Scala" library is).

- Any library that depends on Jackson is likely to cause binary compatibility issues due to the ancient versions shipped with Spark. Guava can be a problem too. Soon enough you'll need to shade a bunch of libraries in your Spark assembly.

- We have a custom sparse matrix implementation that fits our domain well, it was completely broken by the new collections in Scala 2.13. It makes cross-publishing complicated if I don't want to be stuck to Scala 2.12 because of Spark.

Re: From First Principles: Why Scala?

#220

Post by AzzieElbab »

Scala is awesome. It is meant to be a scalable language where you go from level 1 to say 5 as you learn. Problems arise when you put a level 5 guy into a level 1 team and letting him loose. No common style guide, no team training, just write those type signatures approaching tweets max length and let the rest wonder wth is going on
Post reply on HN