Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Towards Scala 3

scala-lang.org

Post reply on HN
Print view View original

151–160 of 383 posts

Re: Towards Scala 3

#151

Post by wenc »

bitL wrote: post #27
kprybol wrote: post #4

Earlier quoted context omitted.

Scala is used pretty heavily in the big data world, particularly if you are working with Spark.

Python is still king there, even DataBricks market it that way. Scala is for advanced stuff that matters.

Scala is still primarily used for data engineering workloads due to the fact it is a JVM language. (There's Java too, but no one wants to write Java code)

PySpark is often used for data science experimentation, but is not as frequently found in production pipelines due to the serialization/deserialization overhead between Python and the JVM. In recent years this problem is less pronounced due to the introduction of Spark dataframes which obviates the performance differences between PySpark and Scala Spark, but for UDFs, Scala Spark is still faster.

A newer development that may change all this is the introduction (in Spark 2.3) of Apache Arrow, a in-memory column store engine which lets Python UDFs work with the in-memory object without serializing/deserializing. This is very exciting as this lets Python get closer to the performance of JVM languages.

I've played around with it on Spark 2.3 -- the Arrow interface works but still not quite production-ready but I expect it will only get better.

Many folks are making strategic bets on Arrow technology due to the AI/GPU craze (and an in-memory standard enables multiple parties to build GPU-based analytics [1]), so there is tremendous momentum there.

At some point I expect the relative importance of Scala on Spark will decrease with respect to Python. (even though Spark APIs are Scala native)

[1] https://www.nextplatform.com/2017/05/09/goai-keeping-databas...

Re: Towards Scala 3

#152

Post by saosebastiao »

nnq wrote: post #2

Just curious, what companies (or kinds of companies) are betting on Scala nowadays? It seems that the people wanting "better Java" all decided they like Kotlin, and the functional programming people now gravitate more towards either F# (for .NET ecosystem) or OCaml/Reason (for the more unixy world). And the academic/research/learning crowd seems to like Haskell more. Who's still in Scala boat? Are Google or Fb or oth…

I still think Scala is a better better-jav than Kotlin, but Kotlin captures that market better because of the much lower learning overhead.

I'm much more of an ML-style programmer, but the thing keeping me away from OCaml is the lack of a multicore runtime. Not only is Scala's concurrency mature and performant, implicits (like having an ExecutionContext) make it extremely easy to use. Even if OCaml released it's multicore runtime today, it would take years to get to the maturity of the Scala ecosystem.

Plus, occasionally I run into areas where OOP really blows away functional programming. It's nice to not have to learn a new language for those occasional times.

Re: Towards Scala 3

#153

Post by YorkshireSeason »

saosebastiao wrote: post #145
YorkshireSeason wrote: post #116

Earlier quoted context omitted.

Haskell is category theory as a language. I disagree with this. Haskell's creation [1, 2] predates the realisation (by the FP community at large) of the close connection between parts of category theory and parts of pure functional programming, which happened in the 1990s, perhaps driven by Moggi's realisation that monads are a fundamental abstraction in computation that can reconcile effects with pure functional com…

> One of Odersky's motives in creating Scala was bringing the power of Haskell into the JVM world, although this is not the only motive (thigh integration of OO and FP being another). I'm not sure why you would think that. Odersky has long claimed ML as his primary functional programming influence, not Haskell. It does have HKT, so there's that...but that's about it. Typeclasses aren't really a part of the language.…

From an abstract PL theory POV, Haskell's key innovation over ML was HKTs. It was an experiment at the time, but one that was successful beyond all expectation: I cannot imagine designing a new PL without HKTs. (Note that Rust is also trying to add HKTs, but has been running into difficulties with type-based lifetime tracking IIRC.)

Implicits are a generalisation of default arguments (I don't know where they were pioneered, maybe C++). Implicits were first tried in Haskell [1]. Scala refined them in several steps, the last being [2] which represents implicits on the type leve. Mimicking type classes via implicits is a well-established Scala idiom [3].

[1] J. R. Lewis, J. Launchbury, E. Meijer, M. B. Shields, Implicit Parameters: Dynamic Scoping with Static Types. https://galois.com/wp-content/uploads/2014/08/pub_JL_Implici...

[2] M. Odersky, A. Biboudis, F. Liu, O. Blanvillain, Simplicitly: Foundations and Applications of Implicit Function Types. https://infoscience.epfl.ch/record/229878/files/simplicitly_...

[3] B. C. d. S. Oliveira, A. Moors, M. Odersky, Type Classes as Objects and Implicits. http://ropas.snu.ac.kr/~bruno/papers/TypeClasses.pdf

Re: Towards Scala 3

#154

Post by actsasbuffoon »

manojlds wrote: post #133
stephen wrote: post #118

...ships in 2020? I'm impressed with what they're doing (and love the language), and it's hard work, and their speed is faster than, say, Java's dead-pace evolution in the 2000s. But, poking around at TypeScript, I've been blown away with the MS/TS speed of development. ~2-3 month release cycles, with non-trivial changes to the language (mapped types, conditional types, etc.), that are themselves unique/novel type sy…

It's not the next release. There is 2.13 and 2.14 to be released. Are you saying Typescript has major version release every 3 months?

Typescript does indeed do frequent releases, though they're not nearly as ambitious as Scala 3.0.

A Typescript release usually has 1-3 exciting new features, and a number of smaller bug fixes and improvements.

Re: Towards Scala 3

#155

Post by coryfklein »

lmm wrote: post #143
coryfklein wrote: post #114

So many cool things already done and even more to come. Some parts that excite me as a Scala nerd: * One can now use implicit function types to basically build your own table language syntax that is type-safe. [1] * Multiversal Equality: you get to decide whether it makes any sense to compare an Apple and an Orange using "==" or "!=", as opposed to Java's forced requirement of allowing you to compare anything with an…

> * Multiversal Equality: you get to decide whether it makes any sense to compare an Apple and an Orange using "==" or "!=", as opposed to Java's forced requirement of allowing you to compare anything with anything. [2] Unfortunately it's being introduced in a fail-unsafe way that as far as I can see makes it virtually useless. If I see "x == y" in code, I have no way to be confident that this isn't an old-fashioned…

Yea, I'd love a compiler switch that enables the strict mode globally, while adding the leniency on a case-by-case basis instead.

Re: Towards Scala 3

#156

Post by JackFr »

nnq wrote: post #2

Just curious, what companies (or kinds of companies) are betting on Scala nowadays? It seems that the people wanting "better Java" all decided they like Kotlin, and the functional programming people now gravitate more towards either F# (for .NET ecosystem) or OCaml/Reason (for the more unixy world). And the academic/research/learning crowd seems to like Haskell more. Who's still in Scala boat? Are Google or Fb or oth…

> It seems that the people wanting "better Java" all decided they like Kotlin

I've been using Java and Scala exclusively for the past 10 years at two large NY banks and two fintech start-ups, and I literally do not know anyone who has ever compiled a Kotlin program.

Simply extrapolating from one's own experience is fraught with peril.

Re: Towards Scala 3

#157

Post by cinnamonheart »

stephen wrote: post #118

...ships in 2020? I'm impressed with what they're doing (and love the language), and it's hard work, and their speed is faster than, say, Java's dead-pace evolution in the 2000s. But, poking around at TypeScript, I've been blown away with the MS/TS speed of development. ~2-3 month release cycles, with non-trivial changes to the language (mapped types, conditional types, etc.), that are themselves unique/novel type sy…

For what it's worth, this isn't just an evolution of the language, but a complete rewrite of the compiler. I appreciate that they're taking the time to do it right. They probably could've pushed new, non-trivial features out faster if they weren't doing a total rewrite.

Re: Towards Scala 3

#158

Post by nikolay_igotti »

pjmlp wrote: post #56
Cyph0n wrote: post #50

Earlier quoted context omitted.

Also, from my understanding, Kotlin is tied to the Intellij ecosystem, so open source tooling is lacking when compared to Scala.

More than that, if you want Kotlin/Native debugging support, you need to either buy Clion or AppCode, as it is not available on the community edition.

It is not correct, debugging via lldb is available.

Re: Towards Scala 3

#159

Post by saosebastiao »

Bizarro wrote: post #26
pjmlp wrote: post #14

Earlier quoted context omitted.

F# only has access to a subset of .NET deployment scenarios, where Scala can be used pretty much everywhere there is a JVM. So most companies actually are more keen to bet on Scala than F#. OCaml/Reason world lacks the wealth of Java libraries, which are an import away on Scala. Regarding Kotlin, so far its killer use case is targeting Android, where one is stuck with an ageing Java subset. Outside of Android, it rem…

I don't think many companies are betting on Scala anymore. F# is just as irrelevant in the grand scheme of things. Let's face it, Kotlin hurts Scala adoption.

That assumes that Java is the only funnel for Scala. Sure, Kotlin likely now captures more of the people trying to get out of Java, but how many people does it capture with backgrounds in Haskell, OCaml, Lisp, R, Python/Pandas, or any other language for that matter?

Re: Towards Scala 3

#160

Post by denisw »

virmundi wrote: post #131

How have people faired with Scala integration with Java libraries? When Scala first came out, I liked the idea. If I wanted to use X library, great, I can. As I pondered it more, this seemed terrible. Say I hug the functional side of Scala. I know what I'm doing. Then I hire a Jr Dev. She comes from the world of Java/Python where functional is no as big of thing (keeping in mind that while Python does support, to a d…

Code reviews. Let developers peer-review each other's code changes and share knowledge about the functional programming practices used in the codebase. This is not a technical problem and has nothing to do with Scala.
Post reply on HN