Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

From First Principles: Why Scala?

lihaoyi.com

Post reply on HN
Print view View original

261–270 of 342 posts

Re: From First Principles: Why Scala?

#261

Post by jolux »

smabie wrote: post #223
lihaoyi wrote: post #130

Earlier quoted context omitted.

I actually came to Scala via F#. You can even find exact dates of those transitions in my stackoverflow history. Scala and F# are sinilar enough to basically be the same language, though I found F# had a number of warts and idiosyncracies that made me move on to Scala. Scala didn't have all of these (though of course it had warts of its own!) and is what stuck with me for the long term

F# has a much more primitive type system than Scala and isn't particularly impressive imo. And it doesn't have functors ala OCaml.

What I like about F# vs Scala is the ML-style syntax and the relative simplicity. I have found Scala staggeringly complex every time I have tried to learn it, it seems like the C++ of functional programming languages. This impression may be misguided.

Re: From First Principles: Why Scala?

#262

Post by AzzieElbab »

yw3410 wrote: post #81
dkarl wrote: post #74

Earlier quoted context omitted.

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

I am continually perplexed at the swallowing of exceptions in certain FP communities. Result types, Either et cetera make it extremely easy to swallow errors by flatMapping thoughtlessly losing the context of where they were - you typically /want/ the call stack when you hit into an exceptional flow.

How do you mean? If your either turned left you should be stuck with that exact left and no amount of flatmapping should effect your stack, it simply does not kick in unless you added something like leftmap that swallows the exception

Re: From First Principles: Why Scala?

#263

Post by valenterry »

u678u wrote: post #257

Old articles of "Why Scala" talked about how great lambdas are, pattern matching, lightweight classes without getters/setters. Now Java does those things the reason to use Scala are few.

That's because these concepts where new and difficult for many people at the time. Now that they already know them from other languages (which is great!) we can directly start with the next batch of helpful techniques in Scala.

Java is still light years behind in terms of productivity through language features. (however, therefore it has higher backwards compatibility, which also helps productivity)

Re: From First Principles: Why Scala?

#264

Post by lmm »

brabel wrote: post #140
dkarl wrote: post #74

Earlier quoted context omitted.

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

I am fairly certain the jury is out on this question: It's always better to write simple, almost dumb code that anyone can understand than to use advanced abstractions from category theory or whatever. Why? Because every single study or anedocte I've ever heard is like yours: adding complex abstractions to code do NOT make it more reliable. But they do make it much harder to modify, understand, and fix. FP tought us…

You should always reach for the simplest solution first (and Haoyi has a great set of guidelines about that: https://www.lihaoyi.com/post/StrategicScalaStylePrincipleofL... ). The point of those complex FP abstractions is to let you keep writing dumb code even when you want to do something fiddly (such as async operations with error recovery). But don't cargo-cult the complex solution if you don't have the complex problem! A plain function whose result is a plain value should be expressed that way.

Re: From First Principles: Why Scala?

#265

Post by lmm »

yw3410 wrote: post #81
dkarl wrote: post #74

Earlier quoted context omitted.

I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…

I am continually perplexed at the swallowing of exceptions in certain FP communities. Result types, Either et cetera make it extremely easy to swallow errors by flatMapping thoughtlessly losing the context of where they were - you typically /want/ the call stack when you hit into an exceptional flow.

The call stack is "magic" built into the language, so as soon as you're not using the "blessed" way of error handling, you lose it and need to rebuild the same functionality "by hand".

I agree that some kind of logical call stack is a very useful thing to have, and I'd recommend implementing something along the lines of https://github.com/lancewalton/treelog that provides it.

Re: From First Principles: Why Scala?

#266

Post by bjarneh »

lihaoyi wrote: post #35
bjarneh wrote: post #32

Earlier quoted context omitted.

Yes it gets better, and the 'fsc' (Fast Scala Compiler) does improve on the situation somewhat; but I've always found it annoyingly slow. It should be mentioned that I haven't looked very hard at that language in many years for that very reason; so things could have improved somewhat. And from what I understand, the next version of Scala (Dotty) is going address this issue (build speed).

The compiler has sped up 3x in the past few years. That's a very big speedup, and the current experience is likely nothing like what you remember

Sounds good, perhaps a language that needs a revisit.

Re: From First Principles: Why Scala?

#267

Post by valenterry »

aynyc wrote: post #15

Is Scala used outside of Spark and Akka? Spark is my company chosen analytic engine, Scala was/is the main language for majority of the production workload. In the last two years or so, more devs are writing PySpark and SparkSQL jobs than Scala.

Yes, totally. We are using Scala for web development, without using spark. Half of our services run on akka, the other half uses typelevel ecosystem.

Re: From First Principles: Why Scala?

#268

Post by valenterry »

MrPowers wrote: post #141
dominotw wrote: post #38

Earlier quoted context omitted.

def mirrors my experience too. Vast majority of spark jobs were easily ported to sql/dbt and the remaining ones are in pyspark. I used to use a lot of scala spark in backend data processing in 2016 but now its almost down to zero. scala is real big impediment to making data processing accessible to general public in your company. order of preference now at my company is, 1. sql 2. pyspark 3. java spark 4. scala spark…

I've rolled out Scala based Spark interfaces to non-programmers in Databricks notebooks, so it's definitely possible, but only if you stick with the basic language features. Here's a more detailed PySpark vs Scala comparison in case folks are interested: https://mungingdata.com/apache-spark/python-pyspark-scala-wh... I think Scala Spark (using 10% of the language features) is a better technical decision (because it p…

Even when Scala is used more or less like python?

Re: From First Principles: Why Scala?

#269

Post by lmm »

dkarl wrote: post #133
yw3410 wrote: post #81

Earlier quoted context omitted.

I am continually perplexed at the swallowing of exceptions in certain FP communities. Result types, Either et cetera make it extremely easy to swallow errors by flatMapping thoughtlessly losing the context of where they were - you typically /want/ the call stack when you hit into an exceptional flow.

As I am slowly acquiring the habits of thought that make it possible to read and write FP code at a reasonable speed, I'm horrified by how much idiomatic Scala FP code relies on projecting certain assumptions onto types and operations that seem, at first glance, to be neutral mathematical abstractions. For example, I find myself hating the Either type, because I feel like there is a socially established convention th…

> For example, I find myself hating the Either type, because I feel like there is a socially established convention that one half of the Either is the type that matters, the value that you want, the value that is the point of the computation you're doing, and the other half is a garbage value that should never be directly handled. So I really feel like I should conform to the convention and reserve Either for cases where one of the possible types doesn't matter. But how often is it true that one side of the Either doesn't matter? People want me to encode success/failure in an Either type, but if I do that, are they going to treat failure with the care it deserves?

There's always a tradeoff between making the happy path clear and making the error handling explicit. The whole point of Either is to be a middle ground between "both cases are equal weight and you handle them by pattern matching" (custom ADTs) and "only the happy path is visible, the error path is completely invisible magic" (exceptions). Given that people in Python or Java tend to use exceptions a lot more than they use datatypes, I'd argue that a typical Scala codebase puts more emphasis on actually handling errors than a typical codebase in other languages.

Where each case really is of equal weight, consider using a custom datatype (it's only a couple of lines: sealed trait A, case class B(...) extends A, case class C(...) extends A) rather than Either.

Re: From First Principles: Why Scala?

#270

Post by lmm »

abhgh wrote: post #160
dmux wrote: post #107

Earlier quoted context omitted.

>...I've often seen it used to add complexity where none is necessary. This reminded me of a great blurb about Niklaus Wirth's approach to languages: >Wirth’s philosophy of programming languages is that a complex language is not required to solve a complex problem. On the contrary, he believes that l languages containing complex features whose purposes are to solve complex problems actually hinder the problem solving…

What makes this quote really interesting is that Wirth was Martin Odersky's (guy who designed Scala) PhD advisor!

Scala is a very simple language at its heart. Unfortunately that empowers developers to write really complicated libraries, leaving application developers in much the same place as if they were using a language with complicated features. (Most application developers can't even tell the difference - most complaints about "Scala is a complex language" turn out to be "I was using a complex library in Scala")
Post reply on HN