I dispute the claim that compiled languages have "heavyweight build setups" or are "inconvenient for small programs". I am constantly writing small Java programs to do minor tasks. I compile and run those programs with Python scripts, not heavyweight IDEs or build tools.
From First Principles: Why Scala?
281–290 of 342 posts
Re: From First Principles: Why Scala?
#282Scala is nice for doing things with Spark compared to say, Java, but I really think that Kotlin can do just as well. The only advantage Scala has with Spark is the syntactic sugar. Once there is an API for Kotlin (beyond the preview version), I will be dropping Scala like a bad habit. Kotlin is also concise, has better IntelliJ support, less "implicit magic", and better Java interop. https://github.com/JetBrains/kotl…
Kotlin has lots of syntax sugar but the semantics are crazy. It's essentially an imperative language where the only way to reason about effects (suspend functions) is by reasoning about control flow. That already makes things like error handling incomprehensible, and it'll be a really bad fit for Spark's way of doing things (where values have to be serializable and shipped around the cluster) - imagine trying to use a suspend function from a Spark anything, it's completely impossible to implement in a sane way.
Re: From First Principles: Why Scala?
#283I'm a big fan of Scala, though I think the sweet spot for Scala is for a small team with pre-existing Scala's expertise. For a one-person project, I definitely use Scala. I used to use Ruby a lot. You can move fast with safety with the caveat that the code quality is harder to manage/control. Contrasting this with golang where they really take the guard rails to the other extreme (e.g. can't compile if a variable is…
We're scaling out Scala to a larger team here at work, and part of the effort is certainly about putting in guardrails. In fact, a lot of it is putting in thr guardrails people like from Go: enforced autoformatting, banning unused variables, fatal warnings, etc. are all just a flip of a switch these days (though we only enforce these things on-merge, people are free to test locally regardless, which I understand is a…
Yup, golang's guardrails are great, but they should be enforced on CI... not while coding locally.
Re: From First Principles: Why Scala?
#284I dispute the claim that compiled languages have "heavyweight build setups" or are "inconvenient for small programs". I am constantly writing small Java programs to do minor tasks. I compile and run those programs with Python scripts, not heavyweight IDEs or build tools.
In my mind, I, perhaps wrongly, equate compiled with - "runs machine code". Java is considered compiled but runs on the JVM. Python also has compilation as one of the steps and runs in its virtual machine yet it is considered interpreted. Sooo... Where do you draw the line?
Regarding your main point, I agree, compiled languages do not necessarily have complicated build setups. But maybe it is tainted by my experience in this field.
Re: From First Principles: Why Scala?
#285Earlier quoted context omitted.
One very public example of a Clojure shop through and through is NuBank out of Brazil, who employs some 700 "Clojure developers" according to them. In fact they are so committed to Clojure they bought Cognitect last year.
Don't bother, I have the feeling randmeerkat will argue that if the team is so big, the language must be not expressive!
I have no gripe with Clojure, I take issue with someone claiming that Amazon, Apple, and Netflix have systems that operate at scale because of it. Clojure is a fine niche language, but it’s not the secret sauce to big tech web scale.
Re: From First Principles: Why Scala?
#286Earlier quoted context omitted.
I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…
"I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre languag…
I have the same feeling with Kotlin. If you stay away from overusing DSLs and over FPing things (traduction, using Arrow or most of it) it is beautiful and easy to maintain and understand even by junior devs.
And my feeling is the same with Clojure and Scala, I'm not a specialist, but code can be written in such a way that I can even contribute to it.
Re: From First Principles: Why Scala?
#287Earlier quoted context omitted.
"I think people can be good programmers and poor software architects at the same time. Sounds like that's what has happened in this case: brilliant abstractions that are misused and result in a buggy product and less productive team." That's fair. I guess if by good programmer we mean, fluency with the language and algorithms / abstractions /types etc then perhaps they qualify. I really meant something more like "sof…
I would guess that it's mainly a question of experience. When you have smart kids fresh out of school, they are going to be eager to use every tool in their toolbox. Eventually (hopefully) they will develop the wisdom to know when to deploy what.
The issue I've seen is that the help venues are overwhelmingly filled with language-"extremists" that have too much time and lead beginners and those looking for help into the rabbit hole. A bit like people that spend more time telling people to TDD-everything than coding...
Re: From First Principles: Why Scala?
#288Earlier quoted context omitted.
Frankly, they are bad programmers. They are smart highly intelligent people who don't have either knowledge or aptitude or willingness to be good programmers. My pet peeve in this business are supposedly good programmers who get praised despite never having actual results. And it is not like they would be rare.
True, and with people switching jobs every 12 to 24 months (at least in the Bay Area) many of them never get to see the outcome of their choices, and don't learn from experience.
The companies in turn don't realize the outcome of their choice either and just consider the inefficiencies to be the normal status... I've never seen such a low productivity than in large companies with a lot of churn... Thankfully they have good people that can hold and patch the walls and are the real not-recognized "heroes"...
Re: From First Principles: Why Scala?
#289Earlier quoted context omitted.
When I was young I though that good writers are the ones who write such complex sentences that nobody can understand. Later I learned that good writers are clear and understandable. I used to write complex code because I found it elegant. Now I write simple code because I enjoy breaking down a complex problem into its simplest form, which for me means I fully understand the problem.
I think the writing metaphor is a good one; we are writing source code for other humans to read after all.
We are also writing code so we can read it ourselves in a few months or years and still understand what it does!
Re: From First Principles: Why Scala?
#290Earlier quoted context omitted.
I'm on exactly that kind of team now. They've been working on a very simple problem for several years now and have an enormous, sophisticated, but still buggy and unstable solution. They're really smart people, and they had to be extremely capable programmers to get this far, but the embarrassing question is, how would a team of mediocre developers have tackled this problem? They would have picked a mediocre language…
Reading this thread is giving me some catharsis. I worked on a large project in a big tech company where the org’s leadership was ideologically determined to build everything in Scala. We ended up with dozens of engineers and failed to deliver basic functionality. The leadership eventually left the company and now they are running a startup with a few of the Scala savants from the old team. They got lots of funding a…
I don't think it is a Scala problem but more of a leadership/organizational problem. If you look at Jane street, they do everything in OCaml which is an interesting choice to say the least, but their stuff works, evolve and they can demonstrate...