I completely agree with their pragmatic approach to Scala. Scala is a very powerful language, and you can do super crazy stuff. My approach to writing Scala code is similar: 1. Write the code like a better-Java/typesafe-Python (that is, imperative style). 2. Glance through the code. For collections, usually there's a more concise approach. 3. Read the code again. If you think what you did in #2 is making the code _to…
Scala at scale at Databricks
11–20 of 174 posts
Re: Scala at scale at Databricks
#12Re: Scala at scale at Databricks
#13Scala is an organizational red flag for me, especially if their recruiters are looking for "scala engineers."
It's a fairly accurate predictor of a company downturn after getting sunk in total complexity and ensuing certain employee churn.
The only reason for Scala adoption was Spark 10 years ago. Scala/Spark engineers were pulling the top buck. With pyspark and other cloud big data options on the scene, no one sane is using scala for this anymore.
Re: Scala at scale at Databricks
#14Scala is an organizational red flag for me, especially if their recruiters are looking for "scala engineers."
And when I say Scala, I mean libraries like akka http, spray-json and slick. I like FP parts of Scala overall.
You would think that Scala 3 released in 2021 would have first class net/http and json that we can get rid of spray json. But no.
Re: Scala at scale at Databricks
#15Scala is an organizational red flag for me, especially if their recruiters are looking for "scala engineers."
Same. It's a fairly accurate predictor of a company downturn after getting sunk in total complexity and ensuing certain employee churn. The only reason for Scala adoption was Spark 10 years ago. Scala/Spark engineers were pulling the top buck. With pyspark and other cloud big data options on the scene, no one sane is using scala for this anymore.
Plus, the cost of teaching Scala to your new hires means they won't be productive for literally _months_
I agree that other big data tooling providers are eating into spark but I'm surprised people are choosing python over Scala for Spark.
Re: Scala at scale at Databricks
#16This is interesting as in taking a different approach to most other Scala shops I’ve experienced so far where die-hard monoid/functors fanatics take over and make the code base completely incomprehensible to outsiders / new hires. This seems to be a much more sensible approach.
Re: Scala at scale at Databricks
#17Re: Scala at scale at Databricks
#18> If I had to describe the style of Scala written at Databricks, I’d put it at 50% Java-ish, 30% Python-ish, 20% functional This is interesting as in taking a different approach to most other Scala shops I’ve experienced so far where die-hard monoid/functors fanatics take over and make the code base completely incomprehensible to outsiders / new hires. This seems to be a much more sensible approach.
They didnt want to explain / help other devs.
Re: Scala at scale at Databricks
#19> If I had to describe the style of Scala written at Databricks, I’d put it at 50% Java-ish, 30% Python-ish, 20% functional This is interesting as in taking a different approach to most other Scala shops I’ve experienced so far where die-hard monoid/functors fanatics take over and make the code base completely incomprehensible to outsiders / new hires. This seems to be a much more sensible approach.
Re: Scala at scale at Databricks
#20There are pros and cons in every tech. Maintaining a large Scala code base is going to be much easier, less to write, less to maintain, easy to refactor.
Don’t use a library if you have no need for it but don’t dismiss it as fanatical unless you have actually compared what the same functionality in your fav tech looks like.
Choose the right tool for the job, don’t aim it at your foot. Get the job done and have a side project for “fun” coding.
Scala is a damn good option for targeting JVM. Don’t blame the tool.