Earlier quoted context omitted.
I worked at a company full of Scala zealots and elites. It was insanely unproductive and people spent a lot of time trying to design around all the floors in Akka, trying to get Akka working in modern cloud computing environments and the like. I think at some stage specialists from some Akka pro shop had to be air dropped in to try fix it all, but it didn't help much. It always made me laugh how well all the services…
My bad experience with Scala has more to do with the FP side (cats and all that). Akka has been quite pleasant to me, so I'm quite surprised by this. Is the business problem not suited to the actor model in the first place?
Scala at scale at Databricks
51–60 of 174 posts
Re: Scala at scale at Databricks
#52I will toot the author's horn for him. He has a fine series of Scala posts on his blog [1] and his book Hands-On Scala Programming [2] is a great introduction to building real applications with Scala so that any experienced developer can understand and extend them. I work at a small company that has been using Scala for 7 years. Some of the prior employees enjoyed playing with advanced language features and writing l…
While easy to understand, I’m constantly surprised to see this type anti-intellectualism. The fact that you don’t have the required prerequisites to understand a codebase doesn’t mean it’s bad. Educate yourself on Category Theory and functional programming techniques and learn to leverage these tools to your advantage. Alternatively you can work in Go where braindead simplicity is the mandated norm.
Re: Scala at scale at Databricks
#53Scala is an organizational red flag for me, especially if their recruiters are looking for "scala engineers."
Re: Scala at scale at Databricks
#54I will toot the author's horn for him. He has a fine series of Scala posts on his blog [1] and his book Hands-On Scala Programming [2] is a great introduction to building real applications with Scala so that any experienced developer can understand and extend them. I work at a small company that has been using Scala for 7 years. Some of the prior employees enjoyed playing with advanced language features and writing l…
While easy to understand, I’m constantly surprised to see this type anti-intellectualism. The fact that you don’t have the required prerequisites to understand a codebase doesn’t mean it’s bad. Educate yourself on Category Theory and functional programming techniques and learn to leverage these tools to your advantage. Alternatively you can work in Go where braindead simplicity is the mandated norm.
I'm not saying FP is bad, it is actually really powerful when it is in the hands of those who understand when and how to use it. The problem with the attitude that you demonstrate in your reply is also not limited to FP: in the era of OOO, there were those that spread the evangelism of design patterns to all places regardless if it made any sense; before that in the 90's there was a group of programmers that liked to generate code until no colleague understood anymore what was happening. And when you tried to tell them that abstractions don't come for free, but with a cost because they make code harder to understand, the answer for the last 25 years has always be the same, although rarely said straight: that it was only hard for -you- to understand, and not for the enlightened master himself.
I have seen more projects fail because of too much unneeded abstraction than by all other causes together. I even have seen companies go almost bankrupt because of projects engineered by lone wolfs where nobody understood the abstractions anymore except for the designer himself, and at some point he himself not really anymore either.
And the problem is that this mindset continues to be cultivated by CS books and conferences. Few people want to read a book that tells them that the secret of being productive as a team depends on mostly the culture in the team and the simplicity of the code, and not on the latest hyped framework, language or paradigm.
Re: Scala at scale at Databricks
#55Earlier quoted context omitted.
While easy to understand, I’m constantly surprised to see this type anti-intellectualism. The fact that you don’t have the required prerequisites to understand a codebase doesn’t mean it’s bad. Educate yourself on Category Theory and functional programming techniques and learn to leverage these tools to your advantage. Alternatively you can work in Go where braindead simplicity is the mandated norm.
Thank you for this reply. It demonstrates exactly the mindset of those few FP die-hards that I have seen wrecking havoc in a couple of companies where I worked for. I'm not saying FP is bad, it is actually really powerful when it is in the hands of those who understand when and how to use it. The problem with the attitude that you demonstrate in your reply is also not limited to FP: in the era of OOO, there were thos…
Agreed, and what did we get? dogmatic decrying of how OOP is completely useless and objectively bad, not too dissimilar from some of the comments on FP and Scala on here.
Re: Scala at scale at Databricks
#56I will toot the author's horn for him. He has a fine series of Scala posts on his blog [1] and his book Hands-On Scala Programming [2] is a great introduction to building real applications with Scala so that any experienced developer can understand and extend them. I work at a small company that has been using Scala for 7 years. Some of the prior employees enjoyed playing with advanced language features and writing l…
Sure, sometimes progress is slower than you'd like, but that's the case in every large monorepo.
Re: Scala at scale at Databricks
#57Earlier quoted context omitted.
Regarding 1, wouldn't Kotlin be a better choice?
I don't think so. Kotlin is an unmaintainable soup of features: it can match all the selling-point examples of Scala but only by adding ad-hoc special cases for each one. (For example, Kotlin has null safety and it lets you write code using errors-as-values style "either" types - but it has two completely separate syntaxes for these things, and so it's impossible to interoperate or reuse code between those two approa…
Are you sure you're not confusing Kotlin with Scala?
> For example, Kotlin has null safety and it lets you write code using errors-as-values style "either" types - but it has two completely separate syntaxes for these things, and so it's impossible to interoperate or reuse code between those two approaches
And that is a problem how? Stick to one style.
> In practice Kotlin codebases still use magical incomprehensible reflection (Spring Boot)
https://github.com/spring-projects-experimental/spring-fu/tr...
> and magical compile-time manipulation (Kapt)
There's nothing magical about it.
Re: Scala at scale at Databricks
#58Just my counterpoint to all the negativity that people invariably feel compelled to pour out in any thread about Scala. Tons of devs use Scala productively without deep dives into monadic code golf, you know.
Re: Scala at scale at Databricks
#59Earlier quoted context omitted.
Spark is really its own world that’s fairly separate from the Scala community at large. What makes it a ”terrible technology”?
Scala moved the needle five years ago and made java better among other languages. Using scala helped me grow as an engineer. in my view, it has no place in an enterprise. I require a mature environment that favors consistency. It boils down to reducing complexity. I can barely handle the complexity growth due to the demands of the business, whether it be software engineering or people and these reasons do not justify…
Re: Scala at scale at Databricks
#60Scala was a terrible technology that was adopted by my organization five years ago and were still paying the price. How to deprecate this at scale? The extent to which i like scala is as a dsl for describing etl jobs with spark and i prefer the RDD api. Engineers are consistently crippled by it, we dont even use shapeless or scalaz. Spec2 is a terrible dsl that offers a shitty vocabular to write unit tests and makes…